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FOREWORD 


The  Army  Research  Institute  Aviation  Research  and  Develop¬ 
ment  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama,  is  an  opera¬ 
tional  unit  of  the  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  Systems  Research  Laboratory  and  provides 
research  support  in  aircrew  training  to  the  U.S.  Army  Aviation 
Center.  Research  is  conducted  in-house  and  is  augmented  by  on¬ 
site  contract  support  as  required.  This  report  documents  con¬ 
tract  work  performed  by  ARIARDA  to  support  the  U.S.  Army  Aviation 
Systems  Command  (AVSCOM)  at  St.  Louis,  Missouri.  The  work  was 
performed  under  the  Memorandum  of  Understanding  between  AVSCOM 
and  ARI,  dated  10  April  1985. 

One  of  the  primary  objectives  of  the  Army's  Manpower  and 
Personnel  Integration  (MANPRINT)  program  is  to  influence  the 
design  of  military  systems  so  that  they  can  be  operated  and 
maintained  in  the  most  cost-effective  and  safest  manner.  Al¬ 
though  progress  has  been  made  in  designing  Army  aviation  systems 
compatible  with  the  capabilities  and  limitations  of  operators, 
little  attention  has  been  paid  to  designing  systems  that  are 
compatible  with  the  capabilities  and  limitations  of  Army  roain- 
tainers.  Methods  and  models  are  needed  to  improve  the  maintain¬ 
ability  design  of  aviation  systems  early  in  their  development. 

This  report  presents  the  findings  of  a  literature  survey  of 
human  factors  methods  and  models  that  might  be  applied  during  the 
early  stages  of  system  development  to  improve  the  maintainability 
design  of  emerging  Army  aviation  systems.  The  survey  supports 
the  Army's  MANPRINT  program.  Once  the  methods  and  models  with 
the  greatest,  potential  are  identified,  it  will  be  possible  to 
evaluate  them  JJiCQugh  follow-on  research. 

I 

The  results  of  this  research  have  been  briefed  to  the  Direc¬ 
tor,  Systems  Research  Laboratory,  ARI,  and  to  the  Deputy  to  the 
Assistant  Deputy  Chief  of  Staff  for  Personnel,  U.S.  Army  MANPRINT 
Directorate,  on  23  March  1989.  The  results  will  be  used  by 
AVSCOM  to  support  the  selection  of  a  methodology  or  model  for 
improving  maintainability  design. 


Technical  Director 
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A  SURVEY  OF  HUMAN  FACTORS  METHODOLOGIES  AND  MODELS  FOR  IMPROVING 
THE  MAINTAINABILITY  DESIGN  OF  EMERGING  ARMY  AVIATION  SYSTEMS 


EXECUTIVE  SUMMARY 


Requirement: 

Increasingly  complex  aviation  systems  are  being  developed  to 
enhance  the  ability  of  Army  aviators  to  fly,  fight,  and  survive 
on  the  modern  battlefield.  In  the  past,  these  systems  often  were 
designed  with  inadequate  regard  for  the  mental  and  physical  capa¬ 
bilities  and  limitations  of  the  soldiers  available  to  operate  and 
maintain  them.  In  response  to  the  Army's  Manpower  and  Personnel 
Integration  (MANPRINT)  initiative,  several  methodologies  and  mod¬ 
els  have  been  developed  or  modified  recently  for  applying  knowl¬ 
edge  about  human  capabilities  and  limitations  to  improve  the 
design  of  emerging  systems.  However,  the  majority  of  this  work 
has  been  directed  toward  the  system  operator  rather  than  the  sys¬ 
tem  maintainer. 

At  the  request  of  the  Army  Research  Institute  Aviation  Re¬ 
search  and  Development  Activity  (ARIARDA) ,  Anacapa  Sciences  re¬ 
searchers  reviewed  the  literature  to  identify  human  factors1 
methodologies  and  models  that  might  be  used  to  improve  the  main¬ 
tainability  of  emerging  Army  aviation  systems.  This  report  pre¬ 
sents  the  findings  of  the  literature  review. 


Procedure : 

A  literature  review  was  conducted  on  the  topics  of  (a)  main¬ 
tenance,  (b)  maintainability  design,  (c)  comparability  methodolo¬ 
gies,  and  (d)  operator  and  maintainer  behavioral  simulation 
models.  The  review  began  with  a  search  of  (a)  the  National  Tech¬ 
nical  Information  Service  data  base,  (b)  the  Defense  Technical 
Information  Center  data  base,  and  (c)  the  cumulative  indexes  for 
selected  scientific  publications.  Approximately  130  relevant 
documents  were  identified  and  reviewed.  The  documents  reviewed 
include  (a)  human  factors  engineering  and  maintainability  engi¬ 
neering  textbooks;  (b)  maintainability  design  guidelines; 

(c)  other  maintainability  literature  reviews;  (d)  technical  re¬ 
ports;  (e)  journal  articles;  (f)  papers  presented  at  professional 
meetings;  (g)  military  standards  and  handbooks;  and  (h)  Army  reg¬ 
ulations,  field  manuals,  and  technical  manuals. 


1The  term  "human  factors"  is  used  in  this  report  in  its 
broadest  sense.  Unless  otherwise  specified,  the  term  encompasses 
all  six  MANPRINT  domains  (i.e.,  Manpower,  Personnel,  Training, 
Human  Factors  Engineering,  System  Safety,  and  Health  Hazards) . 
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Ninety-nine  of  the  documents  reviewed  are  cited  in  this  re¬ 
port.  Forty-five  documents  describe  system  and  equipment  main¬ 
tenance  and  work  directed  toward  improving  system  and  equipment 
maintainability.  Nine  documents  describe  comparability  methodol¬ 
ogies,  18  documents  describe  operator  models,  and  6  documents 
describe  maintainer  models.  The  remaining  21  documents  address 
MANPRINT  issues  and  system  design  requirements. 


Findings : 

To  provide  a  basis  for  evaluating  the  methodologies  and 
models,  the  report  presents  a  brief  overview  of  aviation  main¬ 
tenance  and  the  research  performed  to  improve  maintainer  per¬ 
formance  and  maintainability  design.  Most  of  the  research 
identified  during  the  literature  review  was  performed  or  spon¬ 
sored  by  the  U.S.  Air  Force  or  the  U.S.  Navy. 

Three  comparability  methodologies  and  seven  behavioral 
simulation  models  that  were  judged  to  have  utility  for  improving 
the  maintainability  design  of  emerging  Army  aviation  systems  were 
identified  during  the  literature  review  and  are  discussed  in  this 
report.  The  three  comparability  methodologies  reviewed  are: 

•  Logistic  Support  Analysis  (LSA) , 

•  Hardware  versus  Manpower  (HARDMAN) ,  and 

•  Acquisition  of  Supportable  Systems  Evaluation  Technology 
(ASSET) . 

The  seven  computer  simulation  models  reviewed  are: 

•  Human  Operator  Simulator  (HOS) , 

•  Microcomputer  Systems  Analysis  of  Integrated  Networks  of 
Tasks  (Micro-SAINT) , 

•  Sequiturs  Workload  Analysis  System  (SWAS) , 

•  Task  Analysis  and  Workload  (TAWL) , 

•  Maintenance  Personnel  Performance  Simulation  (MAPPS) , 

•  Crew  Chief,  and 

•  Profile. 

The  HOS,  Micro-SAINT,  SWAS,  and  TAWL  models  were  developed  to 
model  operator  performance.  The  MAPPS,  Crew  Chief,  and  Profile 
models  were  developed  to  model  maintainer  performance.  Current 
exploratory  research  directed  toward  the  development  of  computer 
models  that  combine  expert  system  and  computer-aided  design  (CAD) 
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techniques  to  improve  maintainability  design  during  the  systems 
engineering  process  also  are  discussed. 

All  of  the  methodologies  and  models  reviewed  were  judged  to 
have  some  utility  for  improving  the  maintainability  design  of 
emerging  Army  aviation  systems.  However,  the  utility  of  the 
comparability  methodologies  and  operator  models  is  limited.  The 
utility  of  the  comparability  methodologies  is  limited  because 
they  produce  little  direct  insight  about  design  changes  required 
to  improve  maintainability,  tend  to  perpetuate  poor  maintainabil¬ 
ity  design  features  and  personnel  practices,  rely  extensively  on 
expert  judgment,  and  are  very  time  consuming  and  labor  intensive. 

The  utility  of  the  operator  models  is  limited  because  they 
do  not  account  for  many  of  the  characteristics  that  distinguish 
system  maintenance  from  system  operation.  In  addition,  each  of 
the  models  has  specific  deficiencies  that  limit  utility  for  simu¬ 
lating  maintainer  performance.  Therefore,  it  is  inappropriate  to 
use  the  operator  models,  in  their  present  forms,  to  simulate 
maintenance  performance  and  to  predict  maintainer  workload. 

The  MAPPS  model  was  developed  to  simulate  maintainer  per¬ 
formance  in  the  nuclear  power  plant  environment.  The  extent  to 
which  the  MAPPS  model  may  be  useful  for  improving  the  maintain¬ 
ability  design  of  emerging  aviation  systems  depends  on  the  simi¬ 
larity  between  maintenance  tasks  and  working  conditions  in  a 
nuclear  power  plant  and  those  in  an  Army  aviation  maintenance 
environment.  The  HOS,  Micro-SAINT,  SWAS,  TAWL,  and  MAPPS  models 
will  require  extensive  modifications  to  be  applied  to  Army  avia¬ 
tion  maintainer  tasks. 

The  Crew  Chief  and  Profile  models  were  developed  to  simulate 
maintainer  performance  of  the  two  types  of  activities  that  re¬ 
quire  the  majority  of  maintainers'  time:  (a)  accessing,  remov¬ 
ing,  and  replacing  equipment,  and  (b)  fault  detection  and 
troubleshooting.  Of  the  methodologies  and  models  reviewed.  Crew 
Chief  and  Profile  were  judged  to  have  the  greatest  potential  for 
improving  the  maintainability  design  of  emerging  Army  aviation 
systems . 

The  literature  review  failed  to  reveal  a  comprehensive  meth¬ 
odology  or  model  of  maintainer  performance  and  workload  that  can 
be  used  to  predict  maintainability  during  the  early  stages  of 
Army  aviation  system  development.  One  reason  for  this  shortcom¬ 
ing  is  the  lack  of  a  validated  set  of  criteria  to  guide  the 
development  of  the  methodology  or  model  and  to  use  in  evaluating 
the  methodology's  or  model's  utility.  Ten  criteria,  drawn  from 
the  maintenance  and  maintainability  design  literature,  are  pro¬ 
posed  for  this  purpose. 
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Utilization  of  Findings: 

The  Crew  Chief  and  Profile  models  should  be  evaluated  to  de¬ 
termine  their  utility  for  improving  the  maintainability  design  of 
Army  aviation  systems.  The  HOS,  Micro-SAINT,  SWAS,  TAWL,  and 
MAPPS  models  also  should  be  evaluated  to  determine  the  desirabil¬ 
ity  and  feasibility  of  modifying  them  to  simulate  the  performance 
of  maintainer  tasks  not  addressed  adequately  by  either  the  Crew 
Chief  or  Profile  models  and  to  predict  maintainer  workload. 
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A  SURVEY  OF  HUMAN  FACTORS  METHODOLOGIES  AND  MODELS  FOR  IMPROVING 
THE  MAINTAINABILITY  DESIGN  OF  EMERGING  ARMY  AVIATION  SYSTEMS 


Section  I:  Introduction 

Increasingly  complex  aviation  systems  are  being  developed  to 
enhance  the  ability  of  Army  aviators  to  fly,  fight,  and  survive  on 
the  modern  battlefield.  However,  these  systems  are  often  designed 
with  inadequate  regard  for  the  mental  and  physical  capabilities 
and  limitations  of  the  soldiers  who  will  be  available  to  operate 
and  maintain  them  (Malone,  Heasly,  Waldeisen,  &  Hayes,  1986;  Neal, 
Robinson,  Takacs,  &  Rainwater,  1986;  Robinson,  Deutsch,  &  Rogers, 
1970) .  Although  some  recent  progress  has  been  made  in  designing 
aviation  systems  to  be  compatible  with  the  capabilities  and  limi¬ 
tations  of  operators,  considerably  less  progress  has  been  made  in 
designing  systems  to  be  compatible  with  the  capabilities  and 
limitations  of  Army  maintenance  personnel. 


The  Maintenance  Problem 

The  term  maintenance  refers  to  the  activities  required  to 
retain  an  item  in  or  restore  it  to  an  operational  condition 
(Department  of  Defense,  1981) .  Smith,  Westland,  and  Crawford 
(1970)  reported  that  the  problem  of  maintaining  military  equipment 
in  a  state  of  readiness  grew  to  enormous  proportions  from  1950  to 
1970,  and  that  the  dominant  cause  of  the  maintenance  problem  was 
the  progressive  and  rapid  increase  in  equipment  complexity.  In  a 
recent  review,  Bond  (1987)  concluded  that  the  situation  had  not 
improved,  and  that  a  "persistent  maintenance  crisis"  exists  in  the 
military . 

Examples  of  the  persistent  maintenance  crisis  abound. 

Nawrocki  (1981)  reported  that,  across  all  military  systems,  about 
30%  of  components  originally  diagnosed  as  faulty  were  actually 
operating  correctly;  for  some  equipment  the  incorrect  fault  diag¬ 
nosis  rate  may  have  been  as  high  as  60%.  Halff  (1984)  reported 
that  only  about  one-half  of  the  combat  aircraft  on  a  U.S.  Navy 
carrier  were  able  to  be  flown  off  the  ship  with  all  systems  in  a 
ready  condition. 


Maintenance  Personnel  Shortage 

A  chronic  shortage  of  qualified  maintenance  personnel  con¬ 
tributes  to  the  maintenance  problem.  Potempa,  Lintz,  and  Luckew 
(1975)  noted  that  it  has  become  extremely  costly  to  recruit  and 
train  individuals  to  the  skill  levels  necessary  to  maintain  modern 
systems.  The  current  maintenance  personnel  problems  will  be  exac¬ 
erbated  in  future  years  because  of  the  projected  reduction  in  the 
number  of  military-age  individuals  who  will  be  available  for 
recruitment.  Furthermore,  the  military  services  will  have  to 


1 


compete  with  the  civilian  job  market  for  the  most  capable  indi¬ 
viduals.  Those  who  are  recruited  will  probably  possess  lower 
skills  and  aptitudes  than  current  service  personnel  (Hitchcock, 
Merriman,  Moore,  &  Field,  1982;  Malone  et  al.,  1986;  Zimmerman, 
Butler,  Gray,  &  Rosenberg,  1984). 

Attrition  further  exacerbates  the  maintenance  personnel 
shortage.  A  small  percentage  of  military  maintenance  technicians 
serve  more  than  a  4-year  term  of  enlistment  before  they  either  are 
promoted  to  supervisory  positions  or  leave  the  military  services. 
For  example,  only  40%  of  aviation  maintenance  technicians  in  the 
U.S.  Army  and  in  the  U  S.  Air  Force  reenlist  after  completing 
their  initial  4-year  enlistment  period  (Moore  et  al.,  1987;  U.S. 
Total  Army  Personnel  Command,  1989) .  This  makes  it  extremely 
difficult  for  most  maintenance  technicians  to  achieve  the  skill 
levels  required  to  maintain  modern  aviation  systems  (Smith  et  al., 
1970)  . 


Maintenance  Costs 

The  high  costs  of  maintenance  also  contribute  to  the  mainte¬ 
nance  problem.  Each  year,  approximately  25  -  30%  of  the  annual 
budget  for  the  Department  of  Defense  is  expended  for  maintenance 
of  military  systems.  The  total  maintenance  costs  for  a  piece  of 
equipment  throughout  its  life-cycle  often  exceed  its  acquisition 
costs  (Christensen  &  Howard,  1981;  Nawrocki,  1981;  Rigby  &  Cooper, 
1961) .  Bond  (1987)  estimated  that  maintenance  costs  often  account 
for  the  highest  percentage  of  an  aviation  system's  total  life- 
cycle  costs. 


Design  Deficiencies 

Another  factor  that  contributes  to  the  maintenance  problem  is 
inadequate  system  and  equipment  design.  Potempa  et  al.  (1975) 
reported  that  poor  maintainability  design  was  a  major  contributor 
to  required  repair  time  and  critical  maintainer  errors  for  the 
maintenance  of  U.S.  Air  Force  avionic  components.  More  recently, 
Deibel  (1988)  identified  30  design  characteristics  in  the  U.S.  Air 
Force's  F-16  fighter  that  significantly  increase  the  required 
maintenance  time. 


Approacnes  to  Improving  System  and -Equipment  Maintainability 

Improving  system  and  equipment  maintainability  is  the  most 
likely  solution  to  the  persistent  maintenance  problem.  Main¬ 
tainability  refers  to  the  ability  of  an  item  to  be  retained  in  or 
restored  to  an  operational  condition  (Department  of  Defense, 
1981).  Specifically,  maintainability  refers  to  the  ease  with 
which  equipment  can  be  inspected,  serviced,  repaired,  or  replaced 
under  specified  conditions.  A  system  that  is  easily  maintainable 
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can  be  quickly  restored  to  service  by  maintenance  personnel  with 
the  skill  levels  that  are  likely  to  be  available  in  the  field. 


Smith  et  al.  (1970)  cite  three  potential  approaches  to 
improving  the  maintainability  of  complex  systems:  (a)  improve 
technician  skills  through  training,  (b)  improve  job  performance 
(e.g.,  troubleshooting)  aids,  and  (c)  improve  equipment  design. 
Prior  to  the  mid-1960s,  the  predominant  approach  was  to  provide 
training  for  improving  technician  skills,  with  a  secondary  empha¬ 
sis  on  providing  job  performance  aids.  Little  attention  was  paid 
to  improving  equipment  design.  The  three  approaches  are  briefly 
discussed  in  the  following  paragraphs. 


Improving  Training 

Until  equipment  complexity  became  overwhelming,  training  was 
considered  the  most  effective  method  of  reducing  time  required  for 
system  maintenance  (Cunningham  &  Cox,  1972) .  However,  as  Smith  et 
al.  (1970)  noted,  efforts  to  improve  training  were  unsuccessful  in 
reducing  the  maintenance  problem.  Improved  training  was  espe¬ 
cially  unsuccessful  in  providing  the  skills  required  to  trouble¬ 
shoot  and  locate  malfunctions  in  electronic  equipment,  an  activity 
that  accounted  for  more  than  60%  of  corrective  maintenance  time. 
Despite  improved  training  programs,  many  maintenance  personnel 
failed  to  develop  the  skills  required  to  maintain  the  complex 
equipment.  As  much  as  one-third  of  all  equipment  malfunctions 
were  being  attributed  directly  to  prior  poor  maintenance  or 
improper  application  of  a  maintenance  procedure  (Rigby  &  Cooper, 
1961;  Robinson  et  al . ,  1970). 


Improving.  Job  .Performance..  Aids 

The  second  approach  to  improving  maintainability  is  to  supply 
technicians  with  better  job  performance  aids  (e.g.,  written 
procedural  guides,  built-in  test  equipment  [BITE],  automatic  test 
equipment  [ATE]).  As  Smith  et  al.  (1970)  noted,  periodic  surveys 
indicate  that  improvements  in  troubleshooting  manuals  result  in 
negligible  increases  in  maintenance  effectiveness.  Furthermore, 
even  the  most  accurate  BITE  and  ATE  missed  between  5%  and  10%  of 
system  faults,  thus  requiring  manual  troubleshooting  (Cunningham  & 
Cox,  1972;  Maxion,  1984)  . 

Although  designed  to  speed  maintenance  and  to  compensate  for 
lower  skill  levels  in  the  maintenance  force,  recent  developments 
in  BITE  and  ATE  did  not  provide  a  satisfactory  solution  to  the 
maintenance  problem  (Coppola,  1984).  In  spite  of,  and  in  certain 
cases  because  of  BITE  and  ATE,  serious  maintenance  problems 
persisted,  such  as  high  false  alarm  rates  and  the  unnecessary 
removal  and  replacement  of  functioning  equipment.  These  problems 
resulted  in  excessive  maintenance  times,  wasted  maintenance 
resources,  and  an  increased  burden  on  the  logistics  support  system 
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(Frederickson,  Linquist,  &  Lehman,  1986) .  This  was  particularly 
true  for  electronic  systems,  because  they  imposed  the  greatest 
demands  on  maintenance  resources  (Richardson,  Keller,  Maxion, 
Poison,  &  DeJong,  1985) .  Furthermore,  additional  maintenance 
resources  were  required  to  maintain  the  BITE  and  ATE. 


Improving  System  and  Equipment  Design 

The  third  approach  to  improving  maintainability  is  to  improve 
the  design  of  systems  and  equipment.  Smith  et  al.  (1970), 

Crawford  and  Altman  (1972),  Greenman  (1975),  Potempa  et  al. 

(1975),  and  Robinson  et  al.  (1970)  conclude  that: 

•  equipment  design  is  the  most  important  factor  contributing 
to  maintainability; 

•  logistics  resources,  such  as  tools  and  test  equipment, 
facilities,  and  spare  parts  are  used  more  effectively  when 
maintainability  has  been  designed  into  a  system  from  the 
beginning;  and 

•  data,  methods,  and  models  that  will  provide  human  factors 
engineering  specifications  to  design  engineers  during 
system  development  are  required  to  produce  truly  cost- 
effective  and  maintainable  systems. 

The  Department  of  Defense  (1983c)  recognized  that  manpower 
and  personnel  shortages  will  continue  to  be  severe  unless  the 
maintainability  problem  is  addressed  during  the  design  process  as 
well  as  through  the  more  traditional  approaches  of  improved 
training  and  job  performance  aids.  Current  Department  of  the  Army 
policy  requires  that  maintainability  be  designed  into  systems  and 
equipment  rather  than  introduced  through  post-design  modifications 
suggested  by  test  results,  field  complaints,  or  product  improve¬ 
ment  initiatives  (Department  of  the  Army,  1986,  1987a,  1987b) . 

However,  an  aviation  system's  maintainability  typically  is 
afforded  the  lowest  priority  during  the  design  process,  being 
secondary  to  performance,  weight,  cost,  and  operability.  Further¬ 
more,  efforts  to  increase  system  performance  and  operability  often 
result  in  added  system  complexity,  with  a  concomitant  increase  in 
maintenance  requirements.  By  the  time  maintainability  problems 
are  identified,  changes  for  the  sake  of  efficient  maintenance  are 
often  not  feasible  (Greenman,  1975;  McDaniel  &  Askren,  1985) . 

Only  about  3%  of  a  system's  life-cycle  costs  are  expended 
during  the  concept  exploration  phase  of  the  material  acquisition 
process.  However,  decisions  affecting  approximately  70%  of  the 
system's  total  life-cycle  costs  are  made  during  that  early  phase 
(King  &  Weaver,  1987;  Risser  &  Berger,  1984)  .  Therefore,  it  is 
more  expedient  and  economical  to  improve  system  maintainability 
through  decisions  in  the  early  conceptual  and  design  stages  than 
through  design  changes  later  in  system  development.  One  of  the 
primary  objectives  of  the  Army's  Manpower  and  Personnel 
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Integration  (MANPRINT)  program  is  to  influence  the  design  of 
military  systems  so  that  they  can  be  operated  and  maintained  in 
the  most  cost-effective  and  safest  manner  consistent  with  the 
manpower  structure,  personnel  aptitudes  and  skills,  and  training 
resource  constraints  of  the  Army  (Department  of  the  Army,  1987b) . 


Need 

In  response  to  the  Army’s  MANPRINT  initiative,  several 
methodologies  and  models  have  been  developed  or  modified  for 
applying  knowledge  about  human  capabilities  and  limitations  to  the 
design  of  military  systems  (Kaplan,  1987;  Malone,  Perse,  Heasly,  & 
Kirkpatrick,  1988) .  The  majority  of  this  work  has  been  directed 
toward  the  role  of  the  human  as  a  system  operator  rather  than  as  a 
system  maintainer.  There  is  a  mounting  body  of  evidence  indicat¬ 
ing  that  methodologies  and  models  are  needed  that  can  be  applied 
to  improve  the  maintainability  design  of  aviation  systems  early  in 
their  development. 

At  the  request  of  the  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIARDA) ,  Anacapa  Sciences, 
Inc.,  researchers  reviewed  the  literature  to  identify  human 
factors1  methodologies  and  models  that  might  be  applied  during  the 
early  stages  of  system  development  to  improve  the  maintainability 
design  of  emerging  Army  aviation  systems.  The  literature  review 
supports  the  Army's  MANPRINT  initiative  to  integrate  manpower, 
personnel,  and  training  considerations  into  the  design  of  emerging 
systems  (Department  of  the  Army,  1987b) .  Once  the  methodologies 
and  models  with  the  greatest  potential  are  identified,  it  will  be 
possible  to  evaluate  their  appropriateness  through  follow-on 
research.  This  report  presents  the  findings  of  the  literature 
review . 


Literature  Reviewed 

An  extensive  review  of  the  literature  was  conducted  on  the 
topics  of  (a)  maintenance,  (b)  maintainability  design,  (c)  compa¬ 
rability  methodologies,  and  (d)  operator  and  maintainer  behavioral 
simulation  models.  The  review  began  with  a  search  of  (a)  the 
National  Technical  Information  Service  data  base,  (b)  the  Defense 
Technical  Information  Center  data  base,  and  (c)  the  cumulative 
indexes  for  selected  scientific  publications.  Approximately  130 
relevant  documents  were  reviewed.  The  documents  reviewed  include: 
(a)  human  factors  engineering  and  maintainability  engineering 
textbooks;  (b)  maintainability  design  guidelines;  (c)  other 
maintainability  literature  reviews;  (d)  technical  reports; 


l-The  term  "human  factors"  is  used  in  this  report  in  its  broadest 
sense.  Unless  otherwise  specified,  the  term  encompasses  all  six 
MANPRINT  domains  (i.e.,  Manpower,  Personnal,  Training,  Human 
Factors  Engineering,  System  Safety,  and  Health  Hazards) . 
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(e)  journal  articles;  (f)  papers  presented  at  professional 
meetings;  (g)  military  standards  and  handbooks;  and  (h)  Army 
regulations,  field  manuals,  and  technical  manuals. 


Ninety-nine  of  the  documents  reviewed  are  cited  in  this 
report.  The  number  of  documents  cited  in  each  of  seven  general 
categories  is  summarized  in  Table  1.  The  focus  of  the  literature 
review  was  placed  on  documents  describing  system  and  equipment 
maintenance  and  research  directed  toward  improving  the  design  of 
systems  and  equipment  to  improve  their  maintainability.  It  was 
considered  appropriate  to  review  operator  models  as  well  as 
maintainer  models  for  two  reasons.  First,  more  progress  has  been 
made  in  developing,  implementing,  and  evaluating  operator  models 
than  maintainer  models,  a  fact  that  is  evidenced  by  the  small 
number  of  maintainer  models  available  and  their  generally  low 
level  of  sophistication.  Second,  the  few  maintainer  models  that 
have  been  developed  cover  limited  aspects  of  maintainer  perfor¬ 
mance  (e.g.,  accessibility,  troubleshooting)  and  do  not  specifi¬ 
cally  address  maintainer  workload  and  its  effect  on  performance. 
Accordingly,  operator  models  were  reviewed  to  assess  their  utility 
for  (a)  simulating  certain  aspects  of  maintainer  performance,  and 
(b)  predicting  maintainer  workload  and  its  effect  on  maintainer 
performance . 


Table  1 

Distribution  of  Literature  Reviewed 


Category 

Number 

Documents 

of 

Cited 

Manpower  and  Personnel  Integration 

6 

System  Design  Requirements 

15 

System  and  Equipment  Maintenance 

25 

Maintainability  Design 

20 

Comparability  Methodologies 

9 

Operator  Models 

18 

Maintainer  Models 

6 

Total 

99 
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This  report  is  organized  into  four  sections.  Section  I 
describes  the  background  and  purpose  of  the  literature  review. 
Section  II  presents  the  results  of  the  literature  review  that 
apply  to  aviation  maintenance  and  maintainability  design. 

Section  III  discusses  three  comparability  methodologies  and  seven 
behavioral  simulation  models  (four  operator  models  and  three 
maintainer  models)  that  were  judged  to  have  potential  for 
improving  the  maintainability  design  of  emerging  Army  aviation 
systems.  This  section  also  briefly  summarizes  current  exploratory 
research  directed  toward  the  development  of  computer  models  that 
combine  expert  system  and  computer-aided  design  (CAD)  techniques 
to  improve  maintainability  design  during  the  systems  engineering 
process.  Section  IV  discusses  and  compares  the  relative  utility 
of  the  methodologies  and  models  reviewed  in  Section  III  for 
improving  maintainability  design,  proposes  a  set  of  criteria  that 
should  be  met  by  a  methodology  or  model  of  Army  aviation 
maintainer  performance,  and  identifies  future  research  needs. 
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Section  II:  Maintenance  and  Maintainability  Design 

Section  II  presents  a  brief  overview  of  topics  relevant  to 
aviation  maintenance  and  maintainability  design.  The  objective  is 
to  acquaint  the  reader  with  the  basic  concepts  and  issues  that 
should  be  addressed  by  the  methodologies  and  models  that  will  be 
discussed  in  Section  III.  The  overview  will  address  the  following 
topics : 

•  types  of  maintenance  activities, 

•  organization  of  Army  aviation  maintenance, 

•  measures  of  maintainability, 

•  maintainability  and  reliability, 

•  maintainability  design  guidelines, 

•  maintainability  prediction  procedures,  and 

•  validation  of  maintainability  design  guidelines  and 
prediction  procedures. 


Types  of  Maintenance  Activities 

There  are  three  general  types  of  maintenance  activities:  (a) 
servicing,  (b)  preventive  (scheduled)  maintenance,  and  (c)  correc¬ 
tive  (unscheduled)  maintenance.  Servicing  is  performed  to  keep 
equipment  in  operating  condition  and  involves  such  activities  as 
lubricating,  fueling,  oiling,  and  cleaning.  Preventive  mainte¬ 
nance  is  performed  to  retain  equipment  in  a  specified  condition  by 
providing  systematic  inspection,  detection,  and  prevention  of 
incipient  failures.  Corrective  maintenance  is  performed  to 
restore  equipment  to  a  specified  condition  once  a  failure  has 
occurred.  Although  the  specific  corrective  maintenance  activities 
will  vary  from  situation  to  situation,  five  major  sequential  steps 
are  typically  performed  (U.S.  Army  Materiel  Command,  1972): 

•  recognizing  that  a  malfunction  exists, 

•  localizing  faults  within  the  system  to  a  particular  piece 
of  equipment, 

•  isolating  faults  within  the  piece  of  equipment  to  a 
specific  defective  component  or  part, 

•  repairing  or  replacing  the  defective  component  or  part,  and 

•  checking  out  the  system  and  returning  it  to  service. 


Organization  of  Army  Aviation  Maintenance 

The  Army  employs  a  forward  support  maintenance  concept  and 
organizes  its  maintenance  activities  to  provide  the  battlefield 
commander  with  the  maximum  number  of  safe,  mission  capable  air¬ 
craft.  Maintenance  assets  are  located  as  close  to  the  operating 
force  as  the  tactical  situation  will  allow,  rather  than  being 
located  at  a  fixed  location  on  the  battlefield.  Maintenance 
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activities  are  accomplished  at  the  lowest  organizational  level 
consistent  with  the  tactical  situation  and  the  available  personnel 
skills,  tools,  and  parts  (Payne,  1988) .  The  objective  is  to  pro¬ 
vide  fast,  continuous,  and  reliable  aviation  maintenance  support 
in  the  highly  mobile,  integrated  battlefield  expected  in  future 
combat  situations  (Department  of  the  Army,  1985)  . 

Army  aviation  maintenance  is  further  organized  into  three 
levels  that  correspond  to  the  nature  and  degree  of  difficulty  of 
the  maintenance  functions  required  (e.g.,  install,  repair, 
replace)  and  the  skills  of  the  resident  personnel.  The  three 
levels  are  (a)  aviation  unit  maintenance  (AVUM) ,  also  known  as 
organizational  maintenance;  (b)  aviation  intermediate  maintenance 
(AVIM) ;  and  (c)  depot  maintenance.  Maintenance  functions  are 
assigned  to  one  of  these  three  levels  according  to  the  Maintenance 
Allocation  Chart  (MAC)  for  each  aircraft.  The  MAC  identifies  the 
tools  and  equipment  required  to  perform  maintenance  functions  on 
specific  components  or  assemblies  (Department  of  the  Army,  1988) . 

Organization  by  level  of  maintenance  is  necessary  because  of 
the  demands  for  tactical  deployment  of  the  equipment;  it  also 
provides  for  efficient  use  of  maintenance  personnel  with  different 
skills  and  skill  levels.  Typically,  the  least  skilled  maintenance 
technicians  are  assigned  to  AVUM  units;  individuals  with  higher 
skill  levels  and  more  experience  are  assigned  to  AVIM  units  and 
repair  depots.  A  brief  description  of  the  types  of  activities 
performed  at  each  maintenance  level  is  presented  in  the  paragraphs 
that  follow.  The  description  is  based  on  material  from  Field 
Manual  (FM)  1-500  (Department  of  the  Army,  1985)  and  Army 
Regulation  (AR)  750-1  (Department  of  the  Army,  1988) . 


AVUM  units  perform  high  frequency  maintenance  functions  that 
are  required  to  retain  the  aircraft  in  or  return  it  to  a  fully 
mission  capable  condition  with  little  downtime.  Because  of 
operational  requirements,  there  is  a  greater  need  to  maintain 
equipment  rapidly  at  this  level  compared  to  the  other  maintenance 
levels.  AVUM  activities  include: 

•  performing  preflight,  postflight,  and  daily  aircraft 
inspections; 

•  performing  servicing  and  preventive  maintenance  checks;  and 

•  inspecting,  lubricating,  cleaning,  adjusting,  and  replacing 
line  replaceable  units  (LRUs) . 

In  general,  the  majority  of  maintenance  technicians'  time  in 
the  AVUM  units  is  spent  (a)  gaining  physical  access  to  equipment 
to  perform  servicing,  preventive  maintenance,  or  corrective  main¬ 
tenance;  and  (b)  detecting,  localizing,  and  isolating  faults  using 
BITE,  ATE,  or  manual  troubleshooting  techniques  (Cunningham  &  Cox, 


1972;  McDaniel  &  Askren,  1985;  U.S.  Air  Force  Systems  Command, 
1988)  . 


The  AVUM  capability  normally  is  located  in  an  aviation 
platoon  and  is  limited  by  the  amount  and  complexity  of  available 
facilities  and  ground  support  equipment,  and  the  availability  of 
critical  personnel  skills.  Maintenance  personnel  at  the  unit 
level  normally  do  not  repair  modules  or  components  removed  from 
the  aircraft.  Instead,  defective  modules  or  components  are  either 
discarded  or  transported  to  the  AVIM  units  or  depots  for  repair. 


Aviation  Intermediate  Maintenance 

All  AVIM  units  are  aircraft  maintenance  companies.  They 
support  AVUM  units  by  repairing  selected  items  that  cannot  be 
repaired  at  the  lower  level.  Maintenance  functions  typically 
performed  at  the  AVIM  level  include  inspection,  troubleshooting, 
test,  diagnosis,  repair,  calibration,  adjustment,  and  alignment  of 
aircraft  modules  and  components.  AVIM  units  send  unserviceable 
items  to  depot  facilities  when  they  are  beyond  their  authorized 
capability,  capacity,  or  authority  to  repair. 


Depot  Maintenance 

Depot  maintenance  is  typically  performed  by  civilian 
personnel  at  a  limited  number  of  highly  specialized,  fixed 
facilities  located  away  from  the  operational  area.  Maintenance 
functions  that  do  not  contribute  to  sustaining  air  mobility  are 
assigned  to  depot  maintenance.  Depot  maintenance  functions 
include  overhauling,  making  major  repairs,  and  painting  aircraft. 


Implications  For  Maintainability  Design 

The  design  engineer  should  be  concerned  primarily  with  the 
problems  of  reducing  maintenance  time  at  the  AVUM  level  because  of 
the  operational  need  for  rapid  maintenance.  Therefore,  it  is 
important  to  design  systems  and  equipment  so  that  maintainers  with 
relatively  low  levels  of  skill  and  experience  can  perform  the 
required  maintenance  functions  and  tasks  quickly  and  accurately. 
However,  design  features  that  enhance  the  performance  of 
maintenance  functions  and  tasks  required  at  the  AVIM  and  depot 
maintenance  levels  should  not  be  neglected. 


Measures  of  Maintainability 

The  most  commonly  used  measure  of  maintainability  is  mean- 
time-to-repair  (MTTR) .  MTTR  is  generally  defined  as  the  average 
time  required  for  a  maintenance  technician  with  a  specified  skill 
level  to  locate  and  isolate  a  fault,  repair  or  replace  the  faulty 
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unit,  and  verify  that  the  malfunction  has  been  corrected.  MTTR 
also  has  been  referred  to  as  mean  corrective  maintenance  downtime 
(Crawford  &  Altman,  1972) . 

When  measuring  maintainability,  it  is  customary  to  separate 
actual  repair  time  (i.e.,  the  time  spent  undergoing  inspection, 
repair,  or  servicing)  from  the  time  required  to  perform  other 
maintenance  related  activities  (e.g.,  administrative  processing, 
and  waiting  for  repair  parts,  ground  support  equipment,  and 
technical  inspectors) .  Actual  repair  time  is  the  maintainability 
measure  that  can  be  most  influenced  by  the  equipment  designer. 

As  Crawford  and  Altman  (1972)  and  Christensen  and  Howard 
(1981)  point  out,  time  data  may  not  always  be  the  best  measure  of 
how  well  a  system  has  been  designed  for  maintainability.  There¬ 
fore,  time  should  not  be  the  only  criterion  considered.  Although 
time  is  relatively  easy  to  measure,  it  has  little  diagnostic  value 
to  the  equipment  designer  because  its  sensitivity  to  specific 
maintenance  design  features  is  unknown.  In  addition  to  time,  the 
following  measures  should  be  considered  (Department  of  Defense, 
1983c;  Mahar,  Kane,  Baerthel,  &  Levine,  1983;  Majoros,  1988; 

Towne,  1988) : 

•  the  number  of  maintenance  personnel  and  skill  levels 
required; 

•  the  potential  for  and  the  type  and  frequency  of  maintainer 
error; 

•  the  probability  of  fault  detection; 

•  the  percentage  of  faults  that  can  be  attributed  or  isolated 
to  a  given  equipment  level; 

•  the  number  of  incorrect  diagnoses; 

•  the  number  of  functional  parts  that  are  unnecessarily 
removed;  and 

•  the  amount  of  physical  and  mental  workload. 

Most  of  the  maintainability  measures  that  have  been  developed 
address  requirements  to  perform  corrective  maintenance  on  a  system 
as  quickly  as  possible  at  the  AVUM  level.  This  typically  results 
in  maintenance  concepts  that  emphasize  replacing  faulty  modules  or 
components  and  sending  the  faulty  items  to  a  higher  maintenance 
level  for  repair.  Designers  often  do  not  put  as  much  emphasis  on 
features  that  facilitate  (a)  servicing  and  performing  preventive 
maintenance  on  a  system  or  (b)  repairing  the  modules  or  components 
that  have  been  transported  to  the  AVIM  units  and  depots 
(Cunningham  &  Cox,  1972) . 


Maintainability  and  Re  1 i ahi 1 1 1 y 

A  system  characteristic  closely  related  to  maintainability  is 
reliability.  Reliability  is  the  probability  that  a  system  compo¬ 
nent  can  perform  its  intended  function  for  a  specified  interval 
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under  stated  conditions  (Department  of  Defense,  1981).  The 
standard  measure  of  reliability  is  the  mean-time-between-f ailure 
(MTBF) .  Reliability  is  an  important  factor  in  determining  the 
overall  supportability  of  a  system  because  it  determines  how  often 
maintenance  will  be  required  for  the  components  of  a  system. 
Reliability  and  maintainability  are  both  regarded  as  key  factors 
in  the  design  of  affordable  and  supportable  systems  (Department  of 
Defense,  1983a) . 


Maintainability  Design  Guidelines 

Maintainability  design  guidelines  describe  equipment  design 
features  that  reduce  the  required  maintenance  time  and  the  proba¬ 
bility  of  errors.  Human  factors  researchers  have  been  developing 
guidelines  for  over  30  years  (e.g.,  Cunningham  &  Cox,  1972; 
Department  of  Defense,  1984a;  Folley  &  Altman,  1956;  Rigby, 

Cooper,  &  Spickard,  1961;  Schafer,  Benson,  &  Clausen,  1961;  U.S. 
Army  Missile  Command,  1963) .  Table  2  presents  a  list  of 
guidelines  extracted  from  MIL-STD-1472C,  Human  Engineering 

Criteria  for  Military  Systems.  Equipment . and  Facilities 

(Department  of  Defense,  1984a) .  Although  there  are  no  universally 
accepted  maintainability  design  guidelines,  those  found  in  MIL- 
STD-1472C  are  representative.  In  addition,  MIL-STD-1472C  is  the 
document  most  often  cited  in  requests  for  proposals  for  aviation 
systems .  Unfortunately,  human  factors  researchers  have  not  been 
very  successful  in  convincing  engineers  to  incorporate  the  guide¬ 
lines  during  system  design  (Meister,  1987) . 


Applicability.. To  Aviation  Systems 

The  design  guidelines  provided  in  MIL-STD-1472C  are  intended 
to  apply  to  a  wide  variety  of  systems  and  equipment  and  therefore 
are  often  general,  vague,  and  ambiguous.  Mahar  et  al.  (1983) 
evaluated  the  appropriateness  of  the  guidelines  for  aviation 
systems  and  cited  the  following  major  deficiencies: 

•  The  anthropometric  data  (e.g.,  strength,  reach)  are 
incomplete  or  inaccurate  for  the  population  of  aviation 
maintenance  technicians. 

•  General  aircraft  equipment  location  requirements  (e.g., 
access  panels,  maintenance  servicing  points)  and  inspection 
features  are  not  covered. 

•  Many  design  areas  that  are  related  to  aircraft 
maintainability  (e.g.,  rigging,  armament,  jacking)  are 
excluded . 

•  Requirements  for  different  aviation  hardware  subsystems 
(e.g.,  flight  control,  electronics)  and  for  different 
aviation  maintenance  levels  (e.g.,  unit,  intermediate)  are 
not  addressed  separately. 
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Table  2 


Examples  of  Maintainability  Design  Guidelines 


Category 

Design  Guideline 

Accessibility 

Structural  members  of  units  or  chassis  shall  not 
prevent  access  to  or  removal  of  items. 

Access  Openings 

All  access  covers  that  are  not  completely 
removable  shall  be  self-supporting  in  the  open 
position . 

Cases 

The  proper  orientation  of  an  item  within  its 
case  shall  be  made  obvious,  either  through 
design  of  the  case  or  by  means  of  appropriate 
labels . 

Conductors 

Cables  shall  be  routed  so  as  to  be  readily 
accessible  for  inspection  and  repair. 

Connectors 

Plugs  shall  be  designed  so  that  it  will  be 
impossible  to  insert  a  wrong  plug  into  a 
receptacle . 

Covers 

It  shall  be  made  obvious  when  a  cover  is  not 
secured,  even  though  it  may  be  in  place. 

Failure 

Indications 

Displays  shall  be  provided  to  indicate 
when  equipment  has  failed  or  is  not  operating 
within  tolerance  limits. 

Fasteners 

Whenever  possible,  identical  screw  and  bolt 
heads  shall  be  provided  to  allow  various  panels 
and  components  to  be  removed  with  one  type  of 
tool . 

Handling 

All  removable  or  carried  units  designed  to  be 
removed  and  replaced  shall  be  provided  with 
handles  or  other  suitable  means  for  grasping, 
handling,  and  carrying. 

Mounting 

Field  removable  items  shall  be  replaceable  by 
use  of  nothing  more  than  common  hand  tools. 
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Mahar  et  al.  (1983)  recommended  that  the  guidelines  be 
further  developed  to  provide  adequate  coverage  of  design  param¬ 
eters  and  a  sufficient  amount  of  data  to  allow  trade-offs  between 
alternative  design  concepts  and  hardware  configurations.  They 
also  recommended  that  the  guidelines  provide  demonstrable  quanti¬ 
tative  criteria  that  can  be  verified  during  design  reviews. 


Maintainability  Prediction  Procedures 

Maintainability  prediction  procedures  are  techniques  for 
generating  estimates  of  the  total  repair  time  of  equipment  (e.g., 
mean  corrective  maintenance  time,  total  downtime)  as  early  as 
possible  in  the  design  process.  Maintainability  prediction  is 
useful  for  identifying  problems  and  providing  an  early  assessment 
of  whether  the  predicted  downtime  and  the  required  quantity  and 
quality  of  personnel,  tools,  and  test  equipment  are  consistent 
with  the  system's  operational  requirements  (Cunningham  &  Cox, 
1972)  . 


Five  recommended  maintainability  prediction  procedures  are 
described  in  detail  in  MIL-HDBK-472 ,  Main*- ^ j nability  Prediction 
(Department  of  Defense,  1984b) .  It  i~  oeyond  the  scope  of  this 
report  to  describe  the  maintainability  prediction  procedures  in 
detail.  Instead,  a  brief  summary  of  each  of  the  five  procedures 
is  presented  in  Table  3.  The  Roman  numerals  in  Table  3  match  the 
Roman  numerals  used  to  identify  the  five  procedures  in  MIL-HDBK- 
472.  All  five  procedures  use  a  set  of  equations  to  predict  the 
total  time  required  to  maintain  a  system  within  a  given  period  of 
operation.  The  equations  are  constructed  by  weighting  the  time 
required  to  repair  the  system's  components  by  the  frequency,  or 
rate,  at  which  the  components  are  expected  to  fail.  The  component 
repair  times  are  obtained  from  standard  time-motion  tables  or 
estimated  from  comparable  systems  and  system  components  under 
similar  operational  conditions.  The  failure  rates  are  obtained 
from  engineering  reliability  data  or  estimated  from  comparable 
systems  and  system  components  under  similar  operational 
conditions . 

The  procedures  differ  with  respect  to  the  following  five 
factors : 

•  the  types  of  systems  to  which  they  can  be  applied  (e.g., 
ships  versus  airplanes) ; 

•  the  types  of  equipment  to  which  they  can  be  applied  (e.g., 
electronic  versus  mechanical) ; 

•  the  phases  during  the  design  process  when  they  can  be 
applied  (e.g.,  concept  evaluation  versus  demonstration  and 
validation) ; 

•  the  types  of  information  required  (e.g.,  estimates  of 
elemental  maintenance  task  times,  number  of  replaceable 
components,  number  of  test  points,  physical  layouts, 
functional  diagrams,  tools  and  test  equipment) ;  and 
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Table  3 

Summary  of  MIL-HDBK-472  Maintainability  Prediction  Procedures 


MIL-HDBK-472 

Procedure  Summary 


I  Estimates  total  flightline  downtime  for  airborne 

electronic  and  electromechanical  systems  involving 
modular  replacement .  Requires  standard  elemental  task 
times . 

II  Predicts  the  corrective  and  preventive  maintenance 

downtime  of  shipboard  and  shore  electronic  equipment 
and  systems.  Requires  standard  elemental  task  times 
or  user  estimates  of  required  manhours. 

Ill  Predicts  the  mean  and  maximum  corrective  and 

preventive  maintenance  downtime  for  Air  Force  ground 
electronic  equipment.  Requires  checklists  of  physical 
design,  personnel,  and  support  factors. 

IV  Predicts  the  mean  or  total  corrective  and  preventive 
maintenance  downtime  of  systems  and  equipment. 

Requires  historical  data,  subjective  evaluation, 
expert  judgment,  and  selective  measurement. 

V  Predicts  mean-time-to-repair,  maximum  corrective 

maintenance  time,  and  mean  maintenance  manhours  per 
operating  hour  for  avionics  and  ground  and  shipboard 
electronics.  Predictions  can  be  made  for  the  unit, 
intermediate,  and  depot  maintenance  levels.  Requires 
standard,  elemental  task  times. 


•  the  output  parameters  that  result  from  exercising  the 
procedures  (e.g.,  total  downtime,  mean  preventive  main¬ 
tenance  time,  mean  maintenance  manhours  per  repair,  mean 
maintenance  manhours  per  operational  hour) . 

The  appropriateness  of  a  particular  procedure  depends  primarily  on 
the  type  of  system  and  equipment  and  the  amount  of  information 
that  is  available. 

Existing  maintainability  prediction  procedures  are  cumber¬ 
some,  time  consuming,  labor  intensive,  and  limited  by  the  accuracy 
and  generalizability  of  the  data  obtained  from  comparable  systems 
or  system  components.  Rapid  increases  in  equipment  complexity 
quickly  render  the  historical  maintenance  task  times  required  by 
the  procedures  obsolete.  The  prediction  procedures  are  highly 
dependent  on  mature  design  data  and  therefore  have  limited  utility 
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during  the  early  conceptual  and  design  stages  of  system  develop¬ 
ment  (Cunningham  &  Cox,  1972).  Furthermore,  the  procedures  do  not 
provide  a  means  for  incorporating  maintainer  capabilities  and 
limitations  data  into  the  prediction  process. 


Validation  of  Maintainability  Design  Guidelines 
and  Prediction  Procedures 

Few  attempts  have  been  made  to  validate  maintainability 
design  guidelines  or  prediction  procedures,  particularly  across 
qualitatively  different  types  of  systems  (e.g.,  fixed  wing 
aircraft  versus  helicopters)  and  equipment  (e.g.,  mechanical 
versus  electrical).  The  available  research  suggests  that: 

•  the  validity  of  maintainability  design  guidelines  depends 
on  the  type  of  equipment  and  the  ability,  experience,  and 
motivation  of  maintenance  technicians  (Potempa  et  al., 

1975;  Topmiller  1964)  ; 

•  equipment  design  factors  are  better  predictors  of  required 
maintenance  time  than  are  personnel  or  support  factors 
(Retterer,  Griswold,  McLaughlin,  &  Topmiller,  1965); 

•  the  current  maintainability  prediction  procedures  are 
moderately  accurate  within  a  very  limited  range  of  systems, 
equipment  types,  and  maintenance  technician  populations; 
and 

•  the  accuracy  of  the  prediction  procedures  depends  on  the 
availability  of  the  required  data  at  an  appropriate  level 
of  detail  and  assumes  that  there  will  be  no  major  changes 
in  maintenance  concepts  (i.e.,  how  and  by  whom  maintenance 
is  to  be  performed)  after  the  predictions  are  made 
(Cunningham  &  Cox,  1972;  Smith  et  al . ,  1970)  . 
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Section  III:  Human  Factors  Methodologies  and  Models 


Section  III  describes  three  comparability  methodologies  and 
seven  behavioral  simulation  models  (four  operator  models  and  three 
maintainer  models)  that  were  selected  for  review  and  discussion  in 
this  report  (see  Table  4) .  Three  primary  criteria  were  used  for 
selecting  the  methodologies  and  models.  First,  some  aspect  of  the 
methodology  or  model  was  judged  to  have  potential  for  improving 
the  maintainability  design  of  emerging  Army  aviation  systems. 
Second,  adequate  written  documentation  that  described  the 
methodology  or  model  and  its  intended  applications  was  available 
in  the  open  literature  during  the  time  the  literature  review  was 
conducted.  Third,  an  operational  version  of  the  methodology  or 
model  is  available  for  evaluation  by  the  research  and  development 
community  and  for  implementation  by  the  Army.  The  descriptions  of 
the  methodologies  and  models  presented  in  this  report  were  derived 
from  reviewing  available  written  documentation  and  not  from 
exercising  the  methodologies  or  models. 

The  first  two  subsections  describe  the  three  comparability 
methodologies  and  the  seven  behavioral  simulation  models  listed  in 
Table  4.  For  each  methodology  or  model,  a  brief  overview  is 
provided  that  identifies  the  sponsoring  agency,  the  methodology  or 


Table  4 

Methodologies  and  Models  Reviewed 


Compar-ability  -Methodologies 

Logistic  Support  Analysis  (LSA) 

Hardware  versus  Manpower  (HARDMAN) 

Acquisition  of  Supportable  Systems  Evaluation  Technology 
(ASSET) 

Behavior  -  Simulation.  Models 


Human  Operator  Simulator  (HOS) 

Microcomputer  Systems  Analysis  of  Integrated  Networks  of 
Tasks  (Micro-SAINT) 

Sequiturs  Workload  Analysis  System  (SWAS) 

Task  Analysis  and  Workload  (TAWL) 

Maintenance  Personnel  Performance  Simulation  (MAPPS) 

Crew  Chief 
Profile 
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model's  purpose  and  general  objectives,  and  hardware  and  software 
requirements.  Following  this  is  a  description  of  the  input  data 
requirements,  the  major  steps  or  procedures,  and  the  output  data 
for  the  methodology  or  model.  The  third  subsection  presents  a 
brief  discussion  of  current  exploratory  research  directed  toward 
development  of  computer  models  that  combine  expert  system  and  CAD 
techniques  to  improve  maintainability  design  during  the  systems 
engineering  process. 


Comparability  Methodologies 

A  comparability  methodology  is  defined  in  this  report  as  a 
structured  set  of  activities  that  are  followed  to  predict  the 
logistics  or  human  resource  requirements  (i.e.,  manpower,  person¬ 
nel,  and  training)  of  proposed  military  systems  using  data 
available  from  comparable  predecessor  systems  or  system  compo¬ 
nents.  The  predicted  logistics,  manpower,  personnel,  and  training 
requirements  for  the  proposed  system  or  equipment  are  important 
factors  that  must  be  considered  during  design  for  maintainability. 
The  Logistic  Support  Analysis  (LSA)  and  the  Acquisition  of 
Supportable  Systems  Evaluation  Technology  (ASSET)  methodologies 
are  primarily  concerned  with  maintainer  rather  than  operator 
resources.  In  comparison,  the  Hardware  versus  Manpower  (HARDMAN) 
methodology  is  concerned  with  both  operator  and  maintainer 
resources . 

The  major  difference  between  the  comparability  methodologies 
listed  in  Table  4  and  the  maintainability  prediction  procedures 
reviewed  in  Section  II  is  that  the  comparability  methodologies 
were  developed  to  predict  logistics  and  human  resource  require¬ 
ments  of  a  system,  whereas  the  maintainability  prediction  proce¬ 
dures  were  developed  to  estimate  system  repair  time.  However, 
both  depend  on  the  availability,  accuracy,  and  generalizability  of 
data  from  comparable  predecessor  systems. 


Logistic  Support _An,a 3- y-s is  (LSA,) 

Overview .  LSA  is  a  comparability  methodology  developed  by 
the  Department  of  Defense  (1983a)  for  identifying,  defining, 
analyzing,  quantifying,  and  processing  logistic  support  require¬ 
ments,  including  maintenance.  The  general  objectives  of  LSA  are 
(a)  to  influence  hardware  design,  (b)  to  establish  the  most  effec¬ 
tive  support  concept,  and  (c)  to  define  logistic  support  resource 
requirements . 

Description .  Table  5  identifies  the  five  categories  of  LSA 
tasks  and  the  activities  included  within  each  task  category. 

During  the  LSA  process,  the  logistic  support  resource  requirements 
of  the  system  are  defined  through  an  integrated  analysis  of  all 
maintainer  functions  and  tasks  to  determine  task  frequencies,  task 
times,  and  personnel  and  skill  requirements.  One  of  the  most 
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Table  5 


Logistics  Support  Analysis  (LSA)  Task  Categories  and  Activities 


LSA  Task  Category 

Activities 

Program  Planning 
and  Control 

Provide  for  formal  program  and  review 
actions . 

Mission  and  Support 
Systems  Definition 

Establish  supportability  objectives 
and  related  design  goals,  thresholds, 
and  constraints  through  comparisons 
with  existing  systems  and  analyses  of 
supportability,  cost,  and  readiness. 

Preparation  and 
Evaluation  of 
Alternatives 

Propose  the  best  balance  between  cost, 
schedule,  performance,  and  support- 
ability;  identify  functional  require¬ 
ments  for  maintenance  tasks  and  trade¬ 
offs  (e.g.,  manpower  and  personnel, 
maintenance  level) . 

Determination  of 
Logistic  Support 
Resource  Requirements 

Identify  the  logistic  support  resource 
requirements  of  the  new  system  in  its 
operational  environment (s) ;  develop 
plans  for  post-production  support; 
analyze  maintenance  tasks. 

Supportability 

Assessment 

Assure  that  specified  requirements  are 
achieved  and  deficiencies  are 
corrected. 

useful  outputs  of  the  LSA  process  is  the  Logistic  Support  Analysis 
Record  (LSAR) .  The  LSAR  is  a  subset  of  documentation  that 
contains  detailed  engineering  and  logistic  support  requirements 
data  (Department  of  Defense,  1983b) .  There  are  several  LSAR 
output  summaries  that  provide  information  for  a  maintenance  task 
analysis  and  that  may  be  useful  for  improving  maintainability 
design.  Lobel  and  Mulligan  (1980)  suggest  that  the  most  relevant 
summaries  are  the  following: 

•  Maintenance  Allocation  Summary, 

•  Personnel  and  Skill  Summary, 

•  Support  and  Test  Equipment  Utilization  Summary, 

•  Special  and  Common  Tool  Requirements, 

•  Maintenance  and  Operator  Task  Analysis, 

•  Failure  Mode  Effects  Analysis,  and 

•  Reliability  and  Maintainability  Summary. 
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Overview .  The  HARDMAN  comparability  methodology  is  a 
structured  approach  for  determining  the  human  resource  require¬ 
ments  (i.e.,  manpower,  personnel,  and  training)  during  the 
earliest  phases  of  weapon  systems  development.  HARDMAN  was 
originally  developed  under  U.S.  Navy  sponsorship,  but  has  recently 
been  modified  and  applied  to  predict  Army  human  resource  require¬ 
ments  (Risser  &  Berger,  1984) .  It  is  designed  primarily  to 
support  front-end  analyses  and  is  most  effectively  applied  during 
the  concept  exploration  and  the  demonstration  and  validation 
phases  of  the  materiel  acquisition  process.  An  automated  version 
of  HARDMAN,  HARDMAN  II,  recently  was  employed  during  development 
of  the  Army’s  Forward  Area  Air  Defense  System.  HARDMAN  II  was 
formerly  called  Man-Integrated  Systems  Technology  (MIST)  (Stewert 
&  Shvern,  1988)  . 

The  objectives  of  the  HARDMAN  methodology  are  (a)  to 
determine  human  resource  requirements  (both  operator  and 
maintainer) ,  (b)  to  identify  system  characteristics  that  generate 

excessive  human  resource  requirements,  and  (c)  to  provide  the 
necessary  information  for  conducting  trade-offs  between  manpower, 
personnel,  training,  and  equipment  design  during  the  early  stages 
of  the  systems  acquisition  process  (Mannle  &  Risser,  1984) .  The 
basic  approach  of  the  HARDMAN  methodology  is  comparability 
analysis;  knowledge  about  similar  existing  systems  or  system 
components  is  used  to  project  requirements  for  the  proposed 
systems  or  system  components. 

Description .  The  HARDMAN  methodology  consists  of  five  major 
interrelated  steps  (Zimmerman  et  al.,  1984): 

•  Establish  a  consolidated  data  base  consisting  of  functional 
descriptions  of  the  proposed  system  and  similar  predecessor 
systems,  including  the  associated  inputs,  such  as  hardware 
reliability,  personnel  information,  costs,  and  training 
data . 

•  Determine  the  manpower  and  personnel  requirements  and 
skills  needed  to  operate  and  maintain  the  system. 

•  Determine  the  training  requirements  likely  to  be  imposed  by 
the  proposed  system. 

•  Conduct  an  impact  analysis  to  establish  likely  manpower  and 
training  shortages  and  to  identify  system  characteristics 
that  generate  high  human  resource  costs. 

•  Perform  a  trade-off  analysis  to  alter  features  of  the 
system  (e.g.,  reliability,  accuracy  of  automated  fault 
diagnosis,  manning  requirements,  skill  level  requirements) 
and  to  reduce  or  eliminate  unreasonable  requirements. 

Each  of  these  steps  requires  several  judgments  about  (a)  the 
selection  of  comparable  predecessor  systems  or  system  components, 
(b)  the  identification  and  assembly  of  appropriate  data  on  the 
predecessor  systems,  (c)  the  expected  performance  of  the  proposed 
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system  as  compared  to  predecessor  systems,  and  (d)  the  required 
personnel  skills.  Gaps  in  the  supporting  data  often  exist  at  the 
early  conceptual  and  design  stages  of  system  development.  Expert 
judgment  is  therefore  employed  to  merge  the  various  pieces  of 
information  and  to  generate  comprehensive  personnel  projections 
(Zimmerman  et  al.,  1984). 


Acquisition  of  Supportable  .Systems  Evaluation_TechnQlQ.gy_.(ASSET) 

Overview .  The  ASSET  methodology  is  a  system  of  analytical 
procedures  and  computer  models  that  evaluates  human  resource 
requirements  for  proposed  weapon  systems  throughout  the  acquisi¬ 
tion  process  (Liberati,  Egber,  French,  &  Preidis,  1985)  .  ASSET 
was  developed  under  the  sponsorship  of  the  U.S.  Air  Force  Human 
Resources  Laboratory.  The  models  included  in  ASSET  are  hosted  on 
the  Control  Data  Corporation  CYBER  System,  Wright-Patterson  Air 
Force  Base,  Ohio.  The  conceptual  approach  and  detailed  logic  used 
by  ASSET  are  very  similar  to  that  used  by  HARDMAN. 

ASSET  was  developed  to  accomplish  the  following  objectives: 

•  Estimate  the  human  and  logistics  resources  that  are 
required  to  support  and  operate  the  weapon  system  through¬ 
out  its  life  cycle. 

•  Coordinate  the  development  of  training  programs  and  tech¬ 
nical  manuals  to  ensure  that  complete  and  cost-effective 
maintenance  performance  instruction  is  available  when  the 
weapon  system  is  deployed. 

•  Ensure  that  supportability  and  human  resource  requirements 
are  explicitly  considered  during  the  design  of  the  weapon 
system. 

Description .  The  basic  elements  of  ASSET  are  eight 
analytical  procedures,  eight  computer  models,  and  a  consolidated 
data  base.  The  application  of  ASSET  centers  around  the  analytical 
procedures  that  are  supported  by  the  computer  models.  One  or  more 
of  the  models  may  be  used  to  support  one  or  more  of  the  proce¬ 
dures.  The  consolidated  data  base  provides  the  data  required  for 
the  ASSET  application.  The  ASSET  procedures  and  models  may  be 
applied  together  to  analyze  either  a  complete  weapon  system  or 
specific  system  components. 

The  ASSET  procedures  most  applicable  to  maintainability 
design  are  the  Integrated  Task  Analysis  Procedure,  the  Maintenance 
Action  Network  Procedure,  and  the  Design  Option  Decision  Tree 
Procedure.  The  Integrated  Task  Analysis  Procedure  outlines  the 
requirements  for  tasks  that  must  be  performed  to  operate  and  main¬ 
tain  a  weapon  system.  To  apply  this  procedure,  the  weapon  system 
hardware  components  are  listed  and  described  at  the  level  of 
specificity  that  accounts  for  all  of  the  maintenance  tasks  that 
must  be  performed.  Subsequently,  hardware  elements  and  mainte¬ 
nance  task  steps  are  identified  and  verified,  cues  and 
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accompanying  responses  for  each  maintenance  step  are  described, 
tools  and  equipment  used  to  accomplish  maintenance  tasks  are 
listed,  and  safety  hazards  and  environmental  factors  related  to 
the  maintenance  tasks  are  evaluated. 

The  Maintenance  Action  Network  Procedure  is  used  to  depict 
the  maintenance  flow  of  a  weapon  system  and  to  define  the  input 
data  used  in  the  application  of  ASSET.  The  Maintenance  Action 
Network  uses  an  event  tree  structure  to  describe  the  maintenance 
resources  required  to  restore  a  weapon  system  to  operational 
readiness.  The  network  identifies  the  possible  maintenance 
outcomes  associated  with  each  subsystem  or  component  failure. 

Other  ASSET  models  can  use  this  information  to  compute  the  total 
maintenance  demand  and  the  requirement  for  support  resources 
(e.g.,  crew  size,  skill  categories,  skill  levels,  support  equip¬ 
ment,  and  average  time  required  to  complete  the  tasks  associated 
with  the  event) . 

The  Design  Option  Decision  Tree  Procedure  is  a  graphic  proce¬ 
dure  for  depicting  the  sequence  of  engineering  decisions  required 
to  resolve  a  design  problem.  The  procedure  describes  the  design 
options  available  at  each  decision  point  (Askren  &  Korkan,  1974) . 
This  procedure  may  be  used  to  assess  the  maintainability  as  well 
as  the  performance,  operability,  and  supportability  of  a  weapon 
system.  Examples  of  factors  that  influence  the  decision  options 
are  performance  requirements,  logistics,  weight,  cost,  reli¬ 
ability,  and  development  risk.  In  addition,  the  following  types 
of  human  resources  data  can  be  added: 

•  the  skill  levels  of  personnel  required  to  perform  trouble¬ 
shooting  on  the  equipment, 

•  the  job  speciality  of  the  maintenance  personnel, 

•  the  time  to  troubleshoot  a  failure  on  the  equipment, 

•  the  ease  of  maintaining  the  equipment,  and 

•  the  complexity  of  the  required  tools. 

The  three  procedures  described  above  are  supported  by  the 
Reliability  and  Maintainability  Model.  This  model  provides  infor¬ 
mation  that  enables  the  system  designer  to  identify  high  support 
resource  consumption  areas  in  which  system  design  efforts  can 
reduce  costs.  The  model  defines  a  measure  of  support  resource 
requirements,  evaluates  this  measure  for  each  system  component, 
and  then  ranks  each  system  component  accordingly.  This  identifies 
the  relative  impact  of  each  component  on  support  requirements  and 
focuses  the  designer’s  attention  on  potential  problem  areas. 

When  the  ASSET  methodology  is  exercised  for  a  baseline  or 
alternative  system,  design  features  that  are  high  consumers  of 
manpower,  personnel,  and  logistic  support  resources  can  be  identi¬ 
fied.  This  allows  the  user  to  compare  the  cost  and  resource 
requirements  of  alternative  configurations.  The  application  of 
ASSET  to  a  developing  weapon  system  permits  and  encourages  the 
early  integration  of  design,  logistic  support,  and  operational 
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concepts  so  that  their  combined  influence  may  result  in  a  cost- 
effective,  supportable  system. 


Behavioral  Simulation  Models 

A  behavioral  simulation  model  is  defined  in  this  report  as  a 
representation  of  behavior  and  the  influence  of  behavior  on  real 
world  events  that  allows  simulated  control  of  the  real  world 
events  and  subsequent  prediction  of  their  effects  on  behavior 
(Siegel  &  Wolf,  1981) .  Behavior  simulation  models  attempt  to 
represent  operator  or  maintainer  behavior  statistically  within  the 
system  under  study  and  produce  measures  of  human-system  perfor¬ 
mance  effectiveness  (Lysaght  et  al.,  1988).  Because  of  the 
requirements  to  perform  a  large  number  of  calculations  quickly  and 
to  implement  logical  decision  rules,  behavioral  simulation  models 
are  usually  implemented  on  a  digital  computer. 

The  Human  Operator  Simulator  (HOS) ,  Microcomputer  Systems 
Analysis  of  Integrated  Networks  of  Tasks  (Micro-SAINT) ,  Sequiturs 
Workload  Analysis  System  (SWAS) ,  and  Task  Analysis  and  Workload 
(TAWL)  models  were  developed  specifically  to  model  operator  tasks. 
In  comparison,  the  Maintenance  Personnel  Performance  Simulation 
(MAPPS) ,  Crew  Chief,  and  Profile  models  were  developed 
specifically  to  model  maintainer  tasks. 


Human  Operator  .Simulator  (HOS.) 

Overview .  HOS  is  a  general  purpose  simulation  tool 
originally  developed  under  U.S.  Navy  sponsorship  for  modeling 
human  operators,  systems,  and  the  environment  (Strieb  &  Wherry, 
1979) .  HOS  was  designed  to  predict  system  performance  and 
operator  workload  by  a  dynamic  interactive  simulation  of  the 
environment,  the  hardware/ software  system,  and  the  operator.  A 
family  of  operator  micromodels  are  available  for  developing  a 
simulation.  The  micromodels  contain  algorithms  that  predict  the 
timing  and  accuracy  of  basic  human  cognitive-perceptual,  communi¬ 
cation,  and  psychomotor  actions.  The  most  recent  version  of  HOS, 
HOS-IV,  is  implemented  in  Microsoft  C  on  an  IBM  PC-AT  compatible 
microcomputer  (Harris,  Iavecchia,  Ross,  &  Schaffer,  1987;  Harris, 
Iavecchia,  &  Bittner,  1988) .  HOS-IV  was  developed  under  the 
sponsorship  of  the  U.S.  Army  Research  Institute  for  the  Behavioral 
Sciences  (ARI) . 

Description .  HOS-IV  is  a  rule-based  simulation  model  that 
uses  knowledge  representation  techniques  to  structure  the 
simulation.  This  "top-down"  approach  allows  the  user  to  design 
the  simulation  flow  to  be  independent  of  the  implementation  of 
low-level  simulation  actions  and  micromodels.  HOS-IV  allows  the 
user  to  simulate  the  effects  of  mental  fatigue  on  the  other 
performance  micromodels  and  to  add  to  or  modify  the  existing  human 
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performance  micromodels.  HOS-IV  can  simulate  the  performance  of 
both  sequential  and  concurrent  tasks. 

Four  classes  of  inputs  are  required  for  HOS-IV:  (a)  descrip¬ 
tions  of  the  system  design,  (b)  procedures  for  using  the  system, 

(c)  human  operator  characteristics,  and  (d)  a  mission  scenario. 
During  a  typical  HOS-IV  implementation,  the  analyst  first  deter¬ 
mines  the  allocation  of  functions  between  the  human  operator  and 
the  machine.  Following  this,  the  analyst  describes  the  environ¬ 
ment,  the  hardware  system,  and  the  operator  procedures  and  tactics 
for  interacting  with  the  system  and  for  accomplishing  mission 
goals . 

The  execution  of  HOS-IV  results  in  a  sequence  of  temporal 
operator  decisions  on  the  basis  of  moment-to-moment  mission  events 
and  predefined  tactics  and  procedures.  Once  a  procedure  is 
selected,  HOS-IV  micromodels  are  invoked  to  perform  specific 
actions  (e.g.,  reading  information  from  a  display  or  manipulating 
a  control) .  The  HOS-IV  outputs  include  (a)  a  timeline  of  events 
for  the  operator,  system,  and  environment,  (b)  user-defined 
measures  of  effectiveness,  and  (c)  standard  analyses,  such  as 
action  completion  time,  frequency  of  actions,  time  per  action,  and 
errors . 

HOS-IV  simulates  a  single  seated  operator  who  is  performing  a 
set  of  tasks  during  a  mission  by  observing  displays,  computing, 
making  decisions,  and  manipulating  controls.  HOS-IV  is  best 
suited  for  performing  analyses  in  which  the  operator’s  workstation 
layout  is  the  main  concern  and  where  performance  degradation 
results  primarily  from  congestion  in  reading  displays  and  manipu¬ 
lating  controls  (Wherry,  1976;  Chubb,  Laughery,  &  Pritsker,  1987) . 
Unlike  most  simulation  models,  HOS-IV  does  not  sample  distribu¬ 
tions  of  performance  data;  instead,  it  relies  on  equations 
describing  relationships  between  parameters  and  performance 
outputs  (Meister,  1985) . 


Microcomputer  Systems  Analysis  of  Integrated  Networks  of  Tasks 
(Micro- SAINT) 

Overview .  Micro-SAINT  is  a  microcomputer  adaptation  of  the 
original  SAINT  (Systems  Analysis  of  Integrated  Networks  of  Tasks) 
software  modeling  program  developed  under  the  sponsorship  of  the 
U.S.  Air  Force.  Micro-SAINT  is  an  extension  of  the  Siegel  and 
Wolf  (1969)  simulation  model.  It  is  used  for  building  and 
executing  task  network  models,  specifically  task  network  models  of 
human  operators.  Micro-SAINT  can  be  run  on  an  IBM  PC-AT 
compatible  microcomputer. 

Description .  Task  network  modeling  involves  the 
decomposition  of  system  performance  into  a  series  of  tasks.  Each 
task  has  associated  parameters.  For  example,  tasks  may  require 
time,  consume  resources,  present  opportunities  for  errors,  or 
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require  interaction  with  other  components  of  the  system.  Micro- 
SAINT  permits  the  modeling  of  discrete  or  continuous  tasks  and 
single  or  multiple  operator  tasks  (Laughery  &  Drews,  1985; 
Laughery,  Drews,  Archer,  &  Kramme,  1986) . 

The  sequencing  of  tasks  is  defined  by  constructing  a  task 
network,  determining  what  variables  are  relevant  to  the  modeling 
problem,  and  determining  how  the  variables  are  affected  by  tasks 
in  the  network  (Laughery  et  al . ,  1986).  The  tasks  are  related  to 
each  other  by  precedence  relationships.  The  precedence  relation¬ 
ships  (a)  specify  the  flow  of  tasks  through  the  network  and  (b) 
indicate  tasks  that  can  be  initiated  following  the  completion  of 
previous  tasks  or  other  specified  conditions.  As  individual  tasks 
are  completed  in  the  network,  they  can  modify  later  precedence 
relationships  and  alter  the  flow  of  the  network  (Meister,  1985) . 

Each  task  in  a  Micro-SAINT  task  network  is  associated  with 
other  tasks  and  with  input/output  parameters  that  specify  the 
nature  of  the  predecessor  tasks,  task  characteristics,  and  the 
logic  for  branching  to  other  tasks.  The  time  to  perform  an 
individual  task  can  be  specified  in  terms  of  the  mean  completion 
time  and  the  expected  distribution  of  performance  times.  Tasks 
can  be  prioritized  in  terms  of  their  importance  to  the  system 
mission;  thus,  less  important  mission  tasks  can  be  skipped  if  time 
grows  short  (Meister,  1985) . 

The  primary  outputs  of  a  Micro-SAINT  program  execution  are 
the  time  required  to  perform  the  tasks  in  the  network  each  time 
the  model  is  executed  and  the  distribution  of  times.  In  addition, 
a  "snapshop"  can  be  taken  at  any  time  during  model  execution  that 
indicates  the  value  of  variables  that  are  associated  with  the 
tasks  (Laughery  et  al.,  1986). 


Sequiturs  Workload  Analysis  System  (SWAS) 

Overview .  SWAS  is  a  computer  simulation  model,  developed 
under  the  sponsorship  of  the  Bell  Helicopter  Company,  that 
provides  man-machine  system  performance  predictions  and  identifies 
"bottlenecks"  that  disrupt  successful  task  completion.  The 
bottlenecks  include  intra-  and  inter-operator  processing 
difficulties  and  equipment  delays.  SWAS  also  allows  the  modeler 
to  assess  (a)  individual  differences  in  operator  performance  of 
discrete  or  continuous  tasks  performed  sequentially  or  concur¬ 
rently  and  (b)  the  effect  of  wearing  protective  clothing.  The 
model  possesses  characteristics  of  both  network  models  and 
production  models.  SWAS  is  designed  to  run  on  an  IBM  PC-AT 
compatible  microcomputer  (Holley  &  Parks,  1987) . 

Description .  SWAS  contains  a  simulation  executive  component 
that  interacts  with  four  principal  modules:  the  Monte  Carlo 
module,  the  production  system  module,  the  individual  differences 
module,  and  the  statistics  module.  The  Monte  Carlo  module 
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produces  task  element  performance  times  from  a  probability 
distribution.  Adjustments  to  task  element  performance  times, 
reflecting  intra-operator  timesharing,  are  made  by  (a)  the 
production  system  module,  which  follows  concepts  described  by 
Wickens'  (1984)  multiple  attentional  resources  model,  and  (b)  the 
individual  differences  module,  which  makes  adjustments  on  the 
basis  of  user-defined  runtime  variables.  The  statistics  module 
provides  descriptive  and  inferential  statistics  about  system 
performance  and  workload. 

SWAS  is  both  a  task  database  management  system  and  a  simu¬ 
lation  and  analysis  system.  Tasks  are  selected  and  organized  into 
task  blocks.  Task  blocks  are  then  merged  to  build  a  mission 
segment  file.  During  this  process,  codes  for  continuous  tasks, 
concurrent  tasks,  and  task  precedences  are  assigned.  Subse¬ 
quently,  the  performance/workload  simulation  and  analysis  are 
conducted. 

SWAS  produces  data  about  the  following  parameters: 

•  the  probability  that  the  system  can  successfully  complete 
the  mission  segment, 

•  the  relative  proportions  of  time  required  by  human  opera¬ 
tors  versus  system  equipment  to  perform  mission  segments, 

•  distributions  of  workload  across  the  system,  and 

•  bottlenecks  resulting  from  task  timesharing  overloads  or 
delays  caused  by  waiting  for  other  operators. 


Ia-sJs_&naly.sis  .and  Workload  (TftWLI 

Overview .  TAWL  comprises  a  series  of  related  models  for 
predicting  operator  workload  in  current  and  emerging  Army 
helicopter  systems  that  were  developed  under  the  sponsorship  of 
ARIARDA  (Aldrich  et  al . ,  1988).  Baseline  workload  prediction 
models  were  developed  for  the  Army’s  proposed  light  helicopter 
experimental  (LHX)  (Aldrich,  Szabo,  &  Craddock,  1986) ,  the  AH-64A 
attack  helicopter  (Szabo  &  Bierbaum,  1986) ,  and  the  UH-60A  utility 
helicopter  (Bierbaum,  Szabo,  &  Aldrich,  1987) .  The  models  permit 
the  user  to  compare  workload  estimates  for  a  baseline  system  with 
workload  estimates  for  the  same  system  with  an  alternate  crew 
configuration  or  with  different  automation  options.  The  models 
are  designed  to  run  on  an  IBM  PC-AT  compatible  microcomputer. 

Description .  The  first  step  in  developing  the  models  is  to 
perform  a  mission,  task,  and  workload  analysis  of  the  aviation 
system.  Data  from  the  analysis  are  summarized  on  function 
analysis  worksheets.  The  worksheets  present  verb-object  task 
descriptors  for  each  task,  identify  the  crewmember  performing  each 
task,  and  identify  the  subsystem(s)  associated  with  each  task. 

The  worksheets  also  present  numerical  ratings  of  each  task's 
workload  components  and  an  estimate  of  each  task's  duration. 

Thus,  the  function  analysis  worksheets  provide  a  comprehensive 
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summary  of  the  information  used  to  establish  the  data  bases  for 
the  workload  prediction  models. 

Both  discrete  and  continuous  tasks  and  sequential  and  concur¬ 
rent  tasks  are  included  in  the  models.  Discrete  tasks  are  further 
categorized  as  fixed  or  random.  Mission  requirements  and  condi¬ 
tions  determine  the  duration  of  concurrent  tasks.  Estimated  task 
performance  times  are  obtained  either  directly  from  observations 
of  pilot  performance  in  a  procedures  trainer  or  indirectly  from 
subject  matter  expert  judgments. 

Workload  is  defined  as  the  total  attentional  demand  placed  on 
the  operators  as  they  perform  the  mission  tasks.  Consistent  with 
Wickens ’  (1984)  theory  that  workload  is  a  multidimensional  con¬ 

struct,  five  different  workload  components  (visual,  auditory, 
kinesthetic,  cognitive,  and  psychomotor)  are  defined  for  each 
task.  Workload  is  estimated  by  using  a  verbally  anchored  7-point 
rating  scale. 

The  computer  models  predict  total  workload  experienced  in  the 
performance  of  discrete  and  continuous  tasks  using  information 
from  the  function  analysis  worksheets.  Two  types  of  time-based 
decision  rules  are  used  for  building  the  mission  segments  from  the 
task  data  base.  First,  function  decision  rules  specify  the 
sequence  and  time  for  the  performance  of  each  task  within  each 
function.  Second,  segment  decision  rules  specify  the  sequence  and 
temporal  relationships  for  combining  the  functions  to  form  mission 
segments . 

The  models  use  the  time-based  function  and  segment  decision 
rules  to  place  the  tasks  performed  by  the  operator (s)  on  the 
mission  timeline  at  appropriate  half-second  intervals.  The  models 
produce  estimates  of  total  workload  associated  with  the  perfor¬ 
mance  of  concurrent  and  sequential  tasks.  The  total  workload  for 
concurrent  tasks  is  computed  by  summing  the  workload  component 
ratings  assigned  during  the  task  analysis.  When  excessive  work¬ 
load  occurs,  the  specific  half-second  intervals  are  identified  on 
the  segment  timeline  by  referring  to  the  workload  component  sums. 
The  models  provide  diagnostic  indexes  of  excessive  operator 
workload  for  concurrent  tasks  and  sequences  of  concurrent  tasks. 

Research  is  currently  being  conducted  to  establish  the 
reliability  of  the  workload  rating  scales  and  to  validate  (a)  the 
workload  parameters  used  to  develop  the  models  and  (b)  the  work¬ 
load  predictions  produced  by  the  models.  The  validation  research 
will  be  performed  through  a  series  of  investigations  employing 
part-  and  full-mission  simulation. 


Maintenance  Personnel  Performance  Simulation  (MAPPS) 

Overview .  The  MAPPS  model  is  a  mainframe-level  digital 
computer  simulation  model  of  the  performance  of  a  nuclear  power 
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plant  maintenance  technician  or  a  team  of  technicians.  MAPPS  was 
developed  under  the  sponsorship  of  the  Oak  Ridge  National 
Laboratory.  The  model  considers  personnel,  environmental,  and 
motivational  variables  to  yield  predictions  of  the  performance 
quality  and  required  performance  time  for  the  simulated  nuclear 
power  plant  maintenance  technicians  working  alone  or  as  a  team 
(Siegel,  Bartter,  Wolf,  Knee,  &  Haas,  1984a,  1984b) .  The  model 
was  developed  using  the  results  of  comprehensive  task  analyses  of 
maintenance  technician  and  maintenance  supervisor  positions. 

During  the  task  analyses,  job  incumbents  rated  maintenance  tasks 
on  the  frequency  of  task  performance,  training  time  and  require¬ 
ments,  consequences  of  inadequate  task  performance,  and  cognitive 
and  perceptual-motor  ability  requirements. 

Description .  Four  types  of  input  data  are  provided  for  each 
simulated  maintenance  task:  conditions  of  work,  characteristics 
of  maintenance  technician (s) ,  task  information,  and  subtask 
information.  Task  information  consists  of  data  describing  the 
complete  task.  Subtask  information  describes  the  characteristics 
of  each  subtask  involved  in  task  completion.  The  input  data  can 
be  varied  by  the  individual  type  of  task  or  in  any  combination  of 
the  four  types  of  input  data. 

During  the  simulation  of  a  subtask,  the  effects  of  the 
following  variables  may  be  considered: 

•  difference  between  the  required  cognitive  and  perceptual- 
motor  abilities  and  the  actual  cognitive  and  perceptual- 
motor  abilities  of  the  maintenance  technician; 

•  technician  fatigue  or  fatigue  relief  resulting  from  rest 
breaks ; 

•  time  since  last  task  performance; 

•  stress  induced  by  faulty  communication,  differences  between 
required  and  actual  abilities,  insufficient  time,  or 
radiation; 

•  technician  level  of  aspiration; 

•  supervisor's  performance  expectation; 

•  quality  of  written  support  procedures; 

•  difference  between  required  and  actual  manning  level; 

•  performance  decrement  due  to  high  environmental 
temperature; 

•  equipment  accessibility;  and 

•  organizational  climate  and  error  detection. 

The  variables  interact  with  the  model  in  accordance  with  specified 
rules  of  logic  (e.g.,  an  increase  in  stress  results  in  an  increase 
in  errors)  and  functional  relationships  based  on  the  human  perfor¬ 
mance  literature.  The  model  can  also  simulate  the  performance  of 
troubleshooting,  decision  making,  and  the  donning  and  doffing  of 
protective  garments. 
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As  a  simulation  is  being  conducted,  it  is  possible  to  allow 
the  simulated  maintenance  technician  or  team  to  skip  a  nonessen¬ 
tial  subtask  under  high  stress  levels.  It  is  also  possible  to 
repeat  the  simulation  of  a  subtask,  loop  ahead  or  back  in  a 
subtask  sequence,  or  branch  into  a  new  subtask  sequence  following 
subtask  failure.  The  simulated  procedures  continue  serially  until 
the  subtasks  constituting  a  task  are  completed.  Typically,  a 
number  of  full-task  simulations  are  run  to  smooth  random  effects. 

The  MAPPS  model  produces  a  variety  of  human  performance 
information,  including  quantitative  technician  performance  data, 
areas  of  success  and  failure,  performance  times,  detected  and 
undetected  errors,  and  stress  levels.  The  analytical  results  can 
be  provided  at  several  levels  of  detail  as  specified  by  the  user. 
The  results  are  also  available  in  summary  form. 


Crew  Chief 

Overview .  Crew  Chief  is  a  CAD  model  that  allows  the 
equipment  designer  to  simulate  the  physical  characteristics  and 
limitations  of  an  aircraft  maintenance  technician.  The  model 
allows  the  designer  to  simulate  a  maintenance  activity  on  a 
computer-generated  image  of  an  aircraft  design  and  to  determine  if 
the  required  maintenance  activity  is  feasible.  Crew  Chief  is 
actually  an  expert  system  that  allows  the  designer  to  perform  the 
functions  of  an  expert  ergonomist  detecting  and  correcting 
design-related  maintainability  deficiencies  on  the  spot  prior  to 
the  development  of  a  hardware  mock-up  (McDaniel,  1988;  McDaniel  & 
Askren,  1985) . 

Crew  Chief  was  developed  to  alleviate  maintainability 
problems  usually  created  if  the  designer  lacks  the  following 
information : 

•  feedback  about  previous  maintainability  problems, 

•  three-dimensional  representations  that  allow  visualization 
and  analysis  of  equipment  accessibility  for  maintenance 
tasks, 

•  experience  with  ergonomics  and  statistics,  and 

•  applicable  ergonomics  data  in  a  useable  and  understandable 
format . 

The  model  was  developed  jointly  by  the  U.S.  Air  Force 
Aerospace  Medical  Research  Laboratory  and  the  U.S.  Air  Force  Human 
Resources  Laboratory  at  Wright-Patterson  Air  Force  Base,  Ohio,  as 
a  tool  for  evaluating  the  maintainability  of  Air  Force  aircraft. 

It  is  an  outgrowth  of  the  Computerized  Biomechanical  Man  Model 
(COMBIMAN) .  Crew  Chief  was  designed  to  interface  with  the  three 
most  popular  three-dimensional  CAD  systems  used  by  aerospace 
manufacturers.  Depending  on  the  CAD  system  used,  Crew  Chief 
requires  either  a  mainframe  computer  or  an  engineering  workstation 
computer . 
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Descript  ion .  The  input  for  Crew  Chief  consists  of  the 
graphic  representations  of  a  system  contained  in  a  CAD  data  base. 
Crew  Chief  is  able  to  access  extensive  anthropometric  data  bases 
containing  information  about  the  physical  characteristics  and 
limitations  of  the  maintenance  technician.  The  model  can 
graphically  represent  the  following  physical  characteristics  of 
the  maintenance  technician  and  work  environment: 

•  correct  body  size  and  properties  for  male  and  female 
maintenance  technicians, 

•  encumbrance  of  clothing  and  personal  protective  equipment, 

•  physical  strength  and  mobility  limitations  for  simulated 
working  postures, 

•  physical  access  for  reaching  into  confined  areas, 

•  visual  access  for  direct  and  indirect  viewing,  and 

•  human  strength  capability  for  hand  tool  usage  and  manual 
materials  handling  tasks. 

Crew  Chief  can  also  handle  interactions  of  these  characteristics. 
For  example,  a  Crew  Chief  simulation  may  show  that,  although  a 
maintenance  technician  can  access  the  head  of  a  bolt  with  a 
wrench,  he  or  she  is  unable  to  apply  enough  torque  to  loosen  the 
bolt  because  of  space  and  mobility  limitations. 

The  model  can  simulate  several  common  work  activities,  such 
as  raising,  lowering,  carrying,  holding,  positioning,  reaching, 
moving,  turning,  grasping,  gripping,  inspecting,  pushing,  and 
pulling.  In  addition.  Crew  Chief  can  model  the  following  common 
hand  tools:  screwdrivers,  socket  wrenches,  pliers,  hammers, 
chisels,  saws,  wrenches,  drills,  riveters,  files,  and  scrapers. 

Crew  Chief  is  currently  being  expanded  in  two  different 
areas.  First,  work  is  under  way  to  incorporate  standard  elemental 
task  performance  times  from  time-motion  data  bases  into  the  model 
to  provide  the  designer  with  the  time  estimations  for  removal  and 
replacement  of  aircraft  components.  This  will  allow  the  designer 
to  consider  variables  such  as  posture,  interference,  clothing 
type,  and  human  fatigue  when  estimating  repair  time.  Second,  the 
user  interface  is  being  enhanced  so  that  the  model  can  interact 
with  additional  CAD  programs  (McDaniel,  1988) . 


Profile 

Overview .  Profile  is  a  generic  expert  system  model  of 
troubleshooting  performance  that  can  be  applied  to  several  types 
of  equipment.  The  model  was  developed  under  the  sponsorship  of 
the  Office  of  Naval  Research.  Profile  simulates  the  diagnosis  of 
sample  system  failures  and  generates  a  testing  sequence  for  each 
sample  failure  that  is  representative  of  the  testing  that  would  be 
performed  by  a  qualified  technician.  The  objective  of  the 
diagnostic  strategy  employed  is  to  minimize  a  combined  function  of 
repair  time  and  consumption  of  spare  parts.  The  model  is  intended 
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to  help  the  designer  determine  the  maintainability  implications  of 
a  wide  range  of  equipment  design  alternatives,  including 
packaging,  modularization,  test  point  provisions,  front  panel 
design,  and  the  extent  of  automated  test  facilities  (Towne,  1984; 
Towne  &  Johnson,  1987;  Towne,  Johnson,  &  Bond,  1988)  .  The 
application  of  Profile  requires  a  mainframe  computer. 

Descript  ion .  The  model  is  organized  as  a  highly  structured 
set  of  generic  troubleshooting  rules  and  associated  metrics 
computed  by  specialized  functions.  The  rules  and  metrics  are  the 
result  of  extensive  experimental-  observations  of  human  diagnostic 
performance  and  studies  of  different  diagnostic  strategies.  The 
model  generates  a  detailed  sequence  of  testing  actions  consisting 
of  test  selection,  test  performance,  and  test  interpretation  that 
are  required  to  isolate  a  fault.  Standard  times  for  individual 
maintenance  actions  are  summed  to  provide  a  total  time  to  diagnose 
and  repair  an  item.  This  is  repeated  for  a  large  sample  of 
representative  failures  to  obtain  a  distribution  of  corrective 
maintenance  times.  This  distribution  represents  the  probable 
corrective  maintenance  time  required  by  the  system  design  and  the 
maintenance  conditions. 

Profile  is  configured  to  produce  near  optimal  fault  isolation 
sequences,  and  thus  simulates  the  performance  of  expert  techni¬ 
cians  when  provided  with  perfect  fault-effect  data.  However,  a 
system  that  is  easily  maintained  by  highly  skilled  technicians  may 
result  in  unacceptably  long  repair  times  and  high  error  rates  when 
only  maintenance  technicians  with  average  skills  and  limited 
experience  are  available.  Therefore,  the  model  can  be  configured 
to  simulate  the  performance  of  a  more  typical  technician  popula¬ 
tion  when  provided  with  degraded  fault-effect  data.  In  this 
manner,  the  Profile  model  can  predict  how  differences  in  skills 
and  experience  may  affect  troubleshooting  performance. 

Four  specific  types  of  information  are  required  to  exercise 
the  Profile  model  for  a  particular  system: 

•  a  list  of  the  replaceable  units  and  their  interconnections; 

•  a  list  of  possible  test  points  and  indicators; 

•  the  disassembly  sequences  required  to  gain  access  to 
internal  parts  and  test  points;  and 

•  the  physical  groupings  of  components  into  modules,  boards, 
units,  etc. 

When  the  Profile  model  is  exercised,  the  following  summary 
data  are  provided  to  the  user: 

•  the  mean  time  to  repair  and  the  distribution  of  repair 
times; 

•  an  analysis  of  the  usefulness  of  all  indicators  and  test 
points,  thus  highlighting  maintenance  features  that  are 
redundant  or  of  marginal  value; 
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•  an  analysis  of  false  replacements,  indicating  those 
components  that  are  likely  to  be  consumed  in  quantities 
greater  than  their  failure  rates  would  indicate; 

•  an  identification  of  additional  indicators  and  test  points 
needed  to  discriminate  between  parts  that  produce  identical 
symptoms  under  the  current  design; 

•  a  summary  of  the  types  and  frequencies  of  maintenance 
actions  required  to  resolve  a  sample  of  faults;  and 

•  the  proportion  of  time  spent  performing  the  maintenance 
actions . 

In  addition  to  evaluating  the  maintainability  characteristics  of  a 
system  design,  Profile  may  be  useful  for  (a)  supporting  an 
intelligent  (i.e.,  individualized)  maintenance  training  system, 

(b)  generating  fault  isolation  strategies,  and  (c)  determining  the 
maintenance  time  required  by  various  repair  policies  (Towne  et 
al . ,  1988) . 

In  summary.  Profile  was  designed  to  generate  troubleshooting 
behaviors  very  similar  to  those  of  qualified  technicians  working 
with  adequate  training,  facilities,  and  time  to  resolve  single, 
persisting  faults.  Furthermore,  exercising  the  model  can  help 
identify  design  features  that  either  contribute  to  maintainer 
errors  (e.g.,  false  replacements)  or  require  higher  maintenance 
technician  skill  levels.  Work  was  recently  completed  to  integrate 
Profile  with  the  Mentor-Graphics  CAD  system  (Towne,  1988) . 


Expert  System/CAD  Approaches  to  Maintainability  Design 

Both  Crew  Chief  and  Profile  represent  initial  attempts  to 
combine  the  features  of  expert  systems  and  CAD  techniques .  Both 
models  are  most  productively  applied  during  the  demonstration  and 
validation  phase  or  the  full-scale  development  phase  of  the 
materiel  acquisition  process  (Flegler,  Permenter,  &  Malone,  1987) . 
A  need  exists  to  develop  expert  system/CAD  techniques  that  can  be 
used  earlier  in  the  design  process  and  that  can  help  improve  the 
quality  of  trade-off  decisions  required  during  the  design  of 
aviation  systems. 

The  Logistics  and  Human  Factors  Division  of  the  U.S.  Air 
Force  Human  Resources  Laboratory  at  Wright-Patterson  Air  Force 
Base,  Ohio,  is  currently  sponsoring  exploratory  research  to 
develop  analytical  models,  computer  software,  data  bases,  and  work 
procedures  that  include  maintenance  and  logistics  factors  in  the 
computer-aided  design  of  aviation  systems.  The  objectives  of  this 
research  are  to  understand  the  systems  engineering  process, 
including  design  trade-off  decisions,  and  to  improve  the 
supportability  that  is  designed  into  aviation  systems.  One  of  the 
more  promising  areas  is  the  integration  of  computer-aided 
maintainability  analysis  with  CAD  (Potempa  &  Gentner,  1988) . 
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The  expected  benefits  of  this  research  are  (a)  a  more  effi¬ 
cient  design  process,  (b)  more  explicit  trade-offs  during  system 
design,  and  (c)  better  reliability  and  maintainability  analysis 
techniques  that  can  be  implemented  throughout  the  stages  of 
systems  acquisition  when  critical  design  decisions  are  made 
(Department  of  the  Air  Force,  1988) .  This  research  appears  to 
have  great  potential  benefit  for  improving  the  maintainability 
design  of  Army  aviation  systems  and  should  be  closely  monitored. 
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Section  IV:  Discussion  and  Recommendations 


Section  IV  consists  of  six  subsections.  The  first  two 
subsections  discuss  and  compare  the  utility  of  the  three 
comparability  methodologies  and  the  seven  behavioral  simulation 
models  for  improving  the  maintainability  design  of  emerging  Army 
aviation  systems.  The  third  subsection  discusses  the  need  for  a 
comprehensive  methodology  or  model  of  Army  aviation  maintainer 
performance  and  proposes  a  set  of  evaluation  criteria.  The  fourth 
subsection  presents  the  conclusions  drawn  from  the  comparison  of 
the  comparability  methodologies  and  behavioral  simulation  models. 
The  fifth  subsection  suggests  three  maintainability  research  tasks 
that  can  be  addressed  in  the  future.  The  final  subsection 
summarizes  the  overall  findings  of  the  research. 


Critique  of  the  Comparability  Methodologies 

The  three  comparability  methodologies  discussed  in  Section 
III  provide  the  means  for  estimating  the  logistic  support  or 
manpower,  personnel,  and  training  requirements  of  a  proposed 
system  if  comparable  systems  or  system  components  exist.  Many  new 
systems  do  evolve  from  existing  systems  or  system  components  (U.S. 
Air  Force  Systems  Command,  1988) .  Therefore,  the  three  compara¬ 
bility  methodologies  have  some  utility  for  the  design  of 
evolutionary  systems.  Of  the  three  methodologies  reviewed,  the 
ASSET  methodology  with  its  maintainability-related  analytical 
procedures  (i.e.,  the  Integrated  Task  Analysis  Procedure,  the 
Maintenance  Action  Network  Procedure,  and  the  Design  Option 
Decision  Tree  Procedure)  and  its  Reliability  and  Maintainability 
computer  model  appears  to  have  the  most  utility  for  addressing 
specific  maintainability  issues. 

However,  comparability  methodologies  have  three  disadvantages 
that  limit  their  utility  for  improving  maintainability  design. 
First,  exercising  the  methodologies  produces  little  direct  insight 
about  specific  design  features  that  will  improve  maintainability. 
This  is  particularly  true  when  the  emerging  system  concept  differs 
significantly  from  existing  systems.  Second,  comparative  method¬ 
ologies  tend  to  perpetuate  (a)  poor  maintainability  design  fea¬ 
tures  and  (b)  poor  personnel  and  training  practices  associated 
with  existing  systems.  Third,  comparability  methodologies  rely 
heavily  on  expert  judgment  and  are  very  time  consuming  and  labor 
intensive . 


Critique  of  the  Behavioral  Simulation  Models 

Both  operator  and  maintainer  behavioral  simulation  models 
were  reviewed  as  part  of  this  research.  As  noted  in  Section  I, 
models  developed  specifically  to  simulate  operator  performance  and 
to  predict  operator  workload  were  reviewed  to  assess  the  extent  to 
which  they  might  be  useful  for  (a)  simulating  certain  aspects  of 
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maintainer  performance  and  (b)  predicting  maintainer  workload  and 
its  effect  on  maintainer  performance. 


System  Maintenance  Versus  System  Operation 

System  maintenance  differs  from  system  operation  in  many 
important  ways.  Table  6  presents  a  list  of  task,  performance,  and 
situational  characteristics  of  aviation  system  maintenance  and 
compares  them  with  system  operation.  It  is  reasonable  to  expect 
that  the  utility  of  a  model  for  improving  maintainability  design 
will  depend  on  the  extent  to  which  it  addresses  these  character¬ 
istics.  Differences  between  system  maintenance  and  system 
operation  are  discussed  in  the  paragraphs  that  follow. 

Maintainer  tasks  usually  are  initiated  by  a  system  failure, 
are  discrete  and  sequential,  and  are  seldom  performed  concurrently 
with  other  tasks.  In  comparison,  operator  tasks  are  usually 
initiated  by  the  mission  events,  can  be  discrete  or  continuous, 
and  can  be  performed  sequentially  or  concurrently  with  other  tasks 
(Drury,  Paramore,  Van  Cott,  Grey,  &  Corlett,  1987)  .  Problem 
solving  plays  a  more  important  role  in  system  maintenance  than  in 
system  operation  (Hornick,  Robinson,  Rogers,  &  Sullivan,  1981; 
Malone,  1975) .  In  comparison  to  system  operation,  system 
maintenance  typically  involves  a  greater  variability  of 
environmental  conditions  (e.g.,  temperature,  illumination), 
required  workspace,  and  required  working  postures.  Maintenance 
usually  has  more  stringent  requirements  for  physical 
accessibility,  physical  stre^th,  mobility,  and  tool  manipulation. 
(McDaniel  &  Askren,  1985) . 

The  components  of  workload  and  the  factors  that  contribute  to 
workload  are  somewhat  different  for  maintainers  than  operators. 
Majoros  (1988)  noted  that  aircraft  maintenance  is  usually 
performed  in  time-pressured  environments  in  which  the  maintainer 
must  rapidly  and  accurately  solve  complicated  fault-isolation 
problems  and  repair  problems,  often  while  in  an  awkward  physical 
posture.  Therefore,  he  suggests  that  maintainer  workload  results 
from  the  interaction  of  cognitive  and  physical  task  requirements, 
especially  when  the  tasks  are  cognitively  complex,  novel  or 
infrequently  performed.  In  addition  to  the  cognitive  and  physical 
requirements  proposed  by  Majoros,  it  is  likely  that  maintainer 
workload  also  results  from  visual  and  kinesthetic  task  require¬ 
ments.  In  comparison,  operator  workload  models  generally  assume 
that  operator  workload  results  from  the  combined  effects  of 
visual,  auditory,  kinesthetic,  cognitive,  and  psychomotor 
attentional  demands  that  accumulate  across  concurrently  performed 
tasks,  particularly  when  the  time  required  to  perform  the  task 
exceeds  the  time  available  (Aldrich  et  al.,  1988;  Lysaght  et  al., 
1988)  . 
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Table  6 


Comparison  of  System  Maintenance  and  System  Operation 
Characteristics 


Characteristic 

Maintenance 

Operation 

Task  Initiator 

System  Failure 

Mission  Events 

Types  of  Tasks 

Mostly  Discrete 

Discrete, 

Continuous 

Temporal  Attributes 
of  Tasks 

Often  Sequential, 
Seldom  Concurrent 

Sequential, 

Concurrent 

Problem  Solving 
Requirements 

Moderate  -  High 

Low  -  Moderate 

Environmental 

Conditions 

Highly  Varied 

Relatively  Constant 

Required  Workspace 

Large 

Small 

Required  Postures 

Highly  Varied 

Relatively  Constant 

Physical  Accessibility 
Requirements 

Moderate  -  High 

Low  -  Moderate 

Visual  Accessibility 
Requirements 

Moderate  -  High 

Moderate  -  High 

Physical  Strength 
Requirements 

Moderate  -  High 

Low  -  Moderate 

Mobility  Requirements 

Moderate  -  High 

Low  -  Moderate 

Tool  Manipulation 
Requirements 

Moderate  -  High 

Low 

Workload  Components 

Visual, 

Kinesthetic, 

Cognitive, 

Physical 

Visual, 

Auditory, 

Kinesthetic, 

Cognitive, 

Psychomotor 

Contributors 
to  Workload 

Time  Pressure 

Concurrent  Tasks 

39 


Four  of  the  seven  models  reviewed  (HOS-IV,  Micro-SAINT,  SWAS, 
and  TAWL)  were  developed  specifically  to  model  the  characteristics 
of  operator  performance  and  to  predict  operator  workload.  They 
have  proven  to  be  useful  for  this  purpose.  However,  all  the 
models  lack  the  mechanisms  for  assessing  the  effects  of  most  of 
the  characteristics  of  system  maintenance  listed  in  Table  6.  In 
addition,  each  of  these  models  has  specific  deficiencies  for 
modeling  maintainer  performance  and  assessing  maintainer  workload 
that  further  limit  its  utility. 

For  example,  HOS-IV  is  limited  to  modeling  a  single,  seated 
operator  in  a  fixed  workstation.  Characteristics  of  many  main¬ 
tainer  tasks  (i.e.,  tasks  requiring  coordination  between  multiple 
technicians  or  mobility)  cannot  be  simulated.  The  model 
represents  human  operator  activity  as  a  discrete,  rather  than 
continuous  process,  and  is  limited  in  its  ability  to  account  for 
the  effects  of  individual  differences  in  training  or  skill  levels 
of  maintenance  technicians.  In  addition,  HOS-IV  assumes  that  the 
operator  carries  out  instructions  without  omitting  a  step  or 
making  any  incorrect  decisions  (Meister,  1985) .  This  makes  it 
impossible  to  simulate  the  effects  of  maintainer  errors  on  a 
proposed  design. 

Micro-SAINT  was  developed  as  a  general  system  for  modeling 
discrete  or  continuous  networks  of  tasks.  It  can  model  sequential 
and  concurrent  tasks  involving  single  or  multiple  operators  but 
cannot  model  within-operator  differences  in  task  performance.  The 
user  is  required  to  supply  human  performance  data  to  the 
Micro-SAINT  model.  For  example,  a  recent  implementation  of  Micro- 
SAINT,  the  Workload  Analysis  Aid  (Fontenelle  &  Laughery,  1988) , 
produces  task  workload  estimates  using  benchmark  rating  scales 
developed  by  McCracken  and  Aldrich  (1984) . 

Both  SWAS  and  TAWL  allow  the  modeler  to  simulate  the  perfor¬ 
mance  of  discrete  or  continuous  tasks  performed  sequentially  or 
concurrently.  Both  can  accommodate  multiple  operators  and  can 
access  elemental  task  performance  times.  SWAS  combines 
characteristics  of  network  models  and  rule-based  production 
systems  and  can  simulate  the  effects  of  individual  differences  on 
performance.  TAWL  contains  procedures  for  conducting  systematic 
mission,  task,  and  workload  analysis  procedures  but  cannot 
simulate  the  effects  of  individual  differences  on  performance. 
Neither  model  addresses  the  specific  components  or  effects  of 
maintainer  workload  on  performance. 


Utility  of  the  Maintainer  Models 

The  MAPPS,  Crew  Chief,  and  Profile  models  were  developed 
specifically  to  model  certain  aspects  of  maintainer  performance 
and  therefore  address  more  of  the  maintainer  characteristics 
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listed  in  Table  6  than  do  the  operator  models.  MAPPS  was 
developed  to  simulate  the  performance  of  a  nuclear  power  plant 
maintenance  technician  or  team  of  technicians  and  can  simulate  the 
effect  of  several  variables  that  affect  maintenance  performance  in 
that  work  environment.  Many  of  the  task,  personnel,  and 
environmental  variables  considered  by  MAPPS  (e.g.,  fatigue,  time 
stress,  quality  of  written  procedures,  equipment  accessibility) 
are  common  to  the  aviation  maintenance  environment.  For  these 
reasons,  the  MAPPS  model  should  be  evaluated  further  to  determine 
(a)  the  similarity  between  maintenance  tasks  and  working 
conditions  in  a  nuclear  power  plant  and  those  in  an  Army  aviation 
environment,  and  (b)  the  extent  to  which  the  MAPPS  model  can  be 
used  to  assess  the  maintainability  implications  of  Army  aviation 
system  and  equipment  design. 

Crew  Chief  and  Profile  simulate  the  types  of  maintenance 
activities  that  historically  have  consumed  the  largest  percentage 
of  maintainers'  time.  Crew  Chief  was  developed  specifically  to 
simulate  the  physical  activities  required  for  an  aviation 
maintenance  technician  to  access,  remove,  and  replace  equipment. 
Profile  was  developed  as  a  generic  simulation  of  fault  diagnosis 
performance  and  can  simulate  the  fault  diagnosis  and  trouble¬ 
shooting  activities  required  to  maintain  aviation  systems.  Both 
Crew  Chief  and  Profile  have  been  designed  to  interface  directly 
with  commercial  mainframe  or  graphics  workstation  CAD  programs 
currently  used  by  the  major  aerospace  manufacturers. 


Proposed  Criteria  For  A  Methodology  or  Model 
For  Improving  Maintainability  Design 

The  literature  review  failed  to  reveal  a  comprehensive 
methodology  or  model  of  maintainer  performance  and  workload  that 
can  be  used  to  predict  maintainability  during  the  early  stages  of 
Army  aviation  system  development.  One  reason  for  this  shortcoming 
is  the  lack  of  a  validated  set  of  criteria  that  can  be  used  to 
guide  the  development  of  a  methodology  or  model  and  to  evaluate  the 
methodology's  or  model's  utility.  The  author  suggests  that,  at  a 
minimum,  a  methodology  or  model  should  meet  the  10  general  criteria 
listed  in  Table  7.  The  criteria  were  developed  after  considering 
the  (a)  results  of  the  maintenance  and  maintainability  design 
literature  reviewed  and  discussed  in  Section  II  and  (b)  the 
characteristics  of  system  maintenance  listed  in  Table  6.  The 
criteria  were  derived  primarily  from  Bond,  1987;  Crawford  and 
Altman,  1972;  Cunningham  and  Cox,  1972;  McDaniel  and  Askren,  1985; 
Smith  et  al.,  1970;  and  Towne  et  al.,  1988. 

Table  8  summarizes  a  preliminary  assessment  of  the  extent  to 
which  the  methodologies  and  models  reviewed  and  discussed  in  this 
report  meet  the  10  criteria.  An  "X”  indicates  that  the  method¬ 
ology  or  model  was  judged  to  meet  the  criterion  to  at  least  a 
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Table  7 


Proposed  Evaluation  Criteria  for  a  Methodology  or  Model  For 
Improving  Maintainability  Design 


1.  Does  the  methodology  or  model  address  the  (a)  types  of 
maintenance  tasks,  (b)  physical  strength,  postural, 
mobility,  and  tool  manipulation  requirements,  and  (c) 
variety  of  working  conditions  encountered  in  Army  aviation 
maintenance? 

2.  Does  the  methodology  or  model  address  the  physical  and 
visual  access  requirements  of  a  system  or  equipment? 

3.  Does  the  methodology  or  model  address  the  fault  diagnosis 
(troubleshooting)  requirements  of  a  system  or  equipment? 

4 .  Does  the  methodology  or  model  predict  maintenance  time  and 
errors  at  different  maintenance  levels  (i.e.,  AVUM,  AVIM, 
depot) ? 

5.  Does  the  methodology  or  model  predict  the  effects  of 
individual  differences  in  maintainer  skills  and  experience 
on  system  maintainability? 

6.  Does  the  methodology  or  model  predict  the  maintainability  of 
qualitatively  different  types  of  systems  (e.g.,  fixed  wing 
vs.  helicopters)  and  equipment  (e.g.,  electronic  vs. 
mechanical) ? 

7.  Can  the  methodology  or  model  automatically  apply  valid 
maintainability  design  guidelines  to  a  proposed  system  or 
equipment  at  appropriate  times  during  the  system  design 
process? 

8.  Does  the  methodology  or  model  account  for  the  components  of 
maintainer  workload  and  their  effect  on  maintainer 
performance? 

9.  Are  the  hardware  and  software  required  to  exercise  the 

methodology  or  model  readily  available  to  the  intended  user? 

10.  Will  the  methodology  or  model  be  accepted  and  utilized  by 

designers  of  Army  aviation  systems  and  equipment? 
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moderate  extent.  A  question  mark  (?)  indicates  that  the  method¬ 
ology  or  model  was  judged  to  meet  the  criterion,  but  only  to  a 
limited  extent.  A  dash  ( — )  indicates  that  the  methodology  or 
model  was  judged  not  to  meet  the  criterion.  As  Table  8  indicates, 
only  Crew  Chief  and  Profile  come  close  to  meeting  all  of  the 
evaluation  criteria.  In  their  present  form,  each  cf  the 
methodologies  or  models  is  deficient  in  some  respect. 

The  proposed  criteria  are  not  intended  to  be  exhaustive  or 
definitive.  However,  it  is  hoped  that  they  will  (a)  provide  a 
helpful  start  toward  the  development  of  more  complete,  more 
specific,  and  perhaps  eventually,  quantitative  criteria  for 
evaluating  the  utility  of  maintainability  design  methodologies  or 
models,  and  (b)  encourage  future  research  efforts  to  develop  a 
comprehensive  maintainer  model  that  can  be  used  to  improve  the 
maintainability  design  of  emerging  Army  aviation  systems. 


Conclusions 

Of  the  methodologies  and  models  reviewed,  the  Crew  Chief  and 
Profile  models  were  judged  to  have  the  greatest  potential  for 
improving  maintainability  design.  Crew  Chief  can  be  used  by  the 
designer  (a)  to  simulate  a  physical  maintenance  activity  (e.g., 
accessing,  removing  and  replacing  a  component)  on  a  computer¬ 
generated  drawing  of  an  aircraft  system  and  (b)  to  estimate  if  the 
proposed  activity  is  feasible,  given  the  physical  characteristics 
of  the  target  maintainer  population.  Profile  can  be  used  by  the 
designer  (a)  to  simulate  the  fault  diagnosis  and  troubleshooting 
performance  of  maintenance  technicians  with  the  diagnostic  skills 
expected  in  the  target  maintainer  population  and  (b)  to  estimate 
if  a  specific  design  will  result  in  an  unacceptably  high  repair 
time  or  number  of  false  replacements. 

Therefore,  it  is  recommended  that  primary  emphasis  should  be 
placed  on  evaluating  the  Crew  Chief  and  Profile  maintainer  models 
to  determine  their  appropriateness  for  improving  the  maintain¬ 
ability  design  of  emerging  Army  aviation  systems.  The  evaluations 
should  consist  of  (a)  conducting  working  sessions  with  personnel 
who  have  developed  or  have  an  intimate  knowledge  of  the  models, 

(b)  exercising  the  models  to  simulate  representative  Army  aviation 
maintenance  tasks  and  comparing  the  models'  predictions  with 
actual  maintenance  performance  data,  and  (c)  conducting  working 
sessions  with  equipment  designers  who  use  the  models  to  assess  the 
maintainability  aspects  of  their  system  and  equipment  designs. 

Secondary  emphasis  should  be  placed  on  evaluating  the  MAPPS 
maintainer  model  and  the  HOS-IV,  Micro-SAINT,  SWAS,  and  TAWL 
operator  models  to  determine  if  it  is  both  feasible  and  desirable 
to  modify  them  to  apply  to  Army  aviation  maintainer  tasks.  It  is 
inappropriate  to  use  the  operator  models,  in  their  present  forms, 
to  simulate  maintainer  performance  and  to  predict  maintainer 
workload.  The  utility  of  the  comparability  methodologies  for 
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these  purposes  is  severely  limited  by  the  deficiencies  described 
previously . 


Future  Research  Needs 

As  noted  in  Section  I,  there  has  been  relatively  little 
research  directed  toward  improving  the  maintainability  design  of 
aviation  systems.  Most  of  the  past  maintainability  design 
research  was  performed  or  sponsored  by  the  U.S.  Air  Force  and  the 
U.S.  Navy.  There  is  a  lack  of  research  directed  toward 
maintenance  and  maintainability  design  problems  encountered  in 
Army  aviation  systems.  Some  of  the  research  findings  from  the 
other  services  may  be  applied  to  Army  aviation  maintenance 
problems,  but  there  are  many  problems  that  cannot  be  solved  by 
generalizing  the  results  of  existing  research.  Army  aviation 
maintenance  is  often  performed  on  different  types  of  systems  and 
equipment  and  in  different  environments  and  working  conditions 
than  in  the  Air  Force  and  the  Navy.  Therefore,  a  program  of 
maintainability  research  is  needed  that  will  address  Army  aviation 
systems,  mission  requirements,  and  operational  environments. 

The  following  three  broad  tasks  should  be  addressed  in  a 
program  of  research: 

•  conduct  aviation  maintenance  task  and  workload  analyses, 

•  monitor  current  research  and  development  being  conducted  by 
other  agencies  to  develop  automated  maintainability  design 
aids,  and 

•  review  the  research  literature  on  troubleshooting 
performance . 

These  tasks  are  discussed  in  the  paragraphs  that  follow. 


Conduct  Aviation  Maintenance  Task  and  Workload  Analyses 

Army  aviation  maintenance  tasks  are  determined  by  the 
specific  concepts  followed  during  the  design  of  Army  aviation 
systems  and  equipment .  Maintenance  tasks  that  are  required 
because  one  set  of  design  concepts  were  followed  may  impose  higher 
levels  of  workload  than  maintenance  tasks  that  would  be  required 
if  alternative  concepts  were  followed.  Thus,  estimates  of 
workload  can  provide  useful  feedback  for  maintainability  design 
(Majoros,  1988;  Morris  &  Rouse,  1985). 

Little  is  known  about  the  nature  and  extent  of  maintainer 
workload.  As  discussed  in  Section  IV,  research  suggests  that 
maintainer  workload  may  differ  qualitatively  from  operator 
workload.  An  improved  understanding  of  the  workload  components  of 
Army  aviation  maintenance  tasks  will  help  determine  how  existing 
operator  workload  prediction  methodologies  and  models  might  be 
modified  and  applied  to  maintainability  design  issues.  Such  an 
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understanding  can  be  achieved  through  a  thorough  analysis  of  Army 
aviation  maintenance  tasks. 

Therefore,  it  is  recommended  that  maintenance  task  and 
workload  analyses  be  conducted  for  a  recently  developed  and 
fielded  Army  aviation  system,  such  as  the  AH-64A  advanced  attack 
helicopter.  The  AH-64A  is  the  operational  Army  aircraft  whose 
design  best  illustrates  the  most  recent  maintainability  design 
concepts .  In  addition,  many  of  the  maintenance  tasks  required  for 
the  AH-64A  are  similar  to  maintenance  tasks  required  for  other 
Army  helicopters,  permitting  some  degree  of  generalization  of  the 
task  analysis  results  to  other  Army  aviation  systems. 

The  task  and  workload  analyses  should  be  conducted  for 
selected  tasks  and  subsystems  (e.g.,  electrical,  flight  control) 
at  the  AVUM  and  AVIM  levels,  and,  if  resources  permit,  at  the 
depot  level.  At  a  minimum,  the  task  analyses  should  provide  data 
on  the  following  variables  (Chapanis  &  Shafer,  1988;  Department  of 
Defense,  1979;  Meister,  1985): 

•  conditions  for  task  initiation, 

•  equipment  acted  upon, 

•  required  response  (s), 

•  nature  of  feedback  provided, 

•  task  frequency  and  duration, 

•  information  and  communication  requirements, 

•  evaluations  and  decisions  required  of  the  maintenance 
technician, 

•  characteristic  errors, 

•  required  tools  and  job  aids, 

•  workspace  and  environmental  considerations, 

•  potential  work  hazards,  and 

•  criteria  for  satisfactory  performance. 

The  overall  objectives  of  the  workload  analyses  should  be: 

•  to  determine  the  components  of  maintainer  workload  and  their 
interaction, 

•  to  determine  how  existing  operator  workload  analytical 
techniques  and  measures  might  be  modified  to  apply  to  the 
maintainer, 

•  to  evaluate  the  effect  of  workload  on  maintainer  performance 
and  errors, 

•  to  relate  maintainer  workload  experienced  during  critical 
maintenance  tasks  to  system  design  features,  and 

•  to  serve  as  a  precursor  to  the  development  of  a  model  of 
Army  aviator  maintenance  performance  and  workload. 

ARIARDA  is  currently  developing  methods  and  computer  models 
for  assessing  operator  workload  in  Army  aviation  systems.  Pos¬ 
sibly,  the  operator  workload  expertise  and  experience  could  be 
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applied  directly  to  maintenance  task  and  workload  analyses  without 
additional  literature  review  or  an  extended  period  of  preparation. 
If  the  aviation  maintenance  task  and  workload  analyses  are 
conducted,  preliminary  task  analysis  data  should  be  collected  at 
(a)  selected  Army  aviation  AVUM  and  AVIM  units  and  (b)  the  Army 
Aviation  Logistics  School,  Fort  Eustis,  Virginia.  During  the 
visits  the  researchers  should: 

•  observe  maintenance  activities  and  problems  in  Army 
aviation  maintenance  units; 

•  observe  aviation  maintenance  training  (e.g.,  classroom, 
part-task  trainers,  procedures  trainers,  simulation,  and 
on-aircraft  instruction  and  training) ;  and 

•  survey  the  availability  and  use  of  job  performance  aids. 

Another  source  of  information  that  should  be  considered 
during  the  maintenance  task  and  workload  analyses  are  aircraft 
accident  and  mishap  data  from  the  U.S.  Army  Safety  Center.  For 
example,  Schmitt  (1983)  identified  several  design  features  during 
a  systematic  evaluation  of  the  Naval  Safety  Center  Accident  and 
Mishap  Data  Base  that  contributed  to  maintainer  errors .  The 
information  obtained  from  the  visits  and  the  accident  and  mishap 
data  can  provide  background  material  required  to  perform  the  task 
and  workload  analyses.  Furthermore,  this  information  may  provide 
insights  about  existing  Army  aviation  maintenance  problems  that 
could  be  alleviated  through  improved  systems  design. 


Monitor  Automated  Maintainability  Design.  Aid  Development 

The  results  of  this  literature  review  indicate  that  there  is 
an  immediate  need  for  a  comprehensive  human  factors  research 
program  that  will  identify  analytic  methods  and  develop 
maintainability  design  tools  that  can  be  implemented  during  the 
concept  exploration  phase  of  emerging  aviation  system  development. 
Human  factors  practitioners  must  learn  how  to  translate  their 
knowledge  about  the  capabilities  and  limitations  of  human 
maintainers  into  design  guidelines  or  specifications  that  are 
meaningful  to  aviation  system  and  equipment  designers. 

Little  is  known  about  how  design  engineers  integrate  differ¬ 
ent  sources  of  information,  particularly  human  performance 
information,  when  making  design  decisions.  As  a  group,  design 
engineers  tend  to  assume  that  (a)  whatever  the  characteristics  of 
the  system,  the  human  operator  and  maintainer  will  be  sufficiently 
flexible  to  compensate  for  design  deficiencies,  and  (b)  a  well 
engineered  system  will  automatically  take  into  consideration  human 
capabilities  and  limitations.  Experience  has  demonstrated  these 
assumptions  to  be  inaccurate.  It  is  evident  that  design  engineers 
typically  give  a  low  priority  to,  or  totally  ignore,  abstract, 
general,  human  factors  inputs  (Meister,  1987)  . 
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Therefore,  human  factors  researchers  need  to  develop  analytic 
methods  and  design  aids  that  will  ensure  that  maintainability 
enhancing  features  are  incorporated  into  system  design.  The 
development  of  maintainability  design  aids  will  require  a  better 
understanding  of  (a)  the  content  and  format  of  data  that  system 
engineers  and  designers  can  use  and  (b)  the  kinds  of  information 
about  human  capabilities  and  limitations  that  design  engineers 
need.  For  example,  research  by  Meister  and  Farr  (1967),  Rogers 
and  Armstrong  (1977),  and  Rogers  and  Pegden  (1977)  suggests  that 
designers  are  more  influenced  by  data  presented  in  quantitative, 
graphic,  or  tabular  terms  than  by  data  that  are  qualitative  and 
verbal.  Meister,  Sullivan,  Finley,  and  Askren  (1969)  found  that 
design  engineers  are  likely  to  relate  certain  design  concepts  and 
characteristics,  such  as  number  and  location  of  test  points  and 
troubleshooting  procedures,  to  the  skill  level  of  maintenance 
technicians . 

In  the  future,  most  aviation  system  design  activities  will  be 
performed  using  CAD  systems  (McDaniel  &  Askren,  1985) .  A  method¬ 
ology  or  model  that  directly  interfaces  with  CAD  systems  and  gives 
the  design  engineer  immediate  feedback  about  the  maintainability 
implications  of  a  specific  design  is  more  likely  to  influence  the 
designer  to  incorporate  maintainability  enhancing  features. 

Coppola  (1984)  proposes  that  an  ideal  program  for  improving  main¬ 
tainability  would  require  the  creation  of  (a)  a  set  of  decision 
rules  for  incorporating  maintainability  design  features  and 
(b)  procedures  for  trading  the  maintainability  design  decision 
rules  off  against  other  system  design  considerations.  Coppola 
notes  that  the  development  of  maintainability  design  decision 
rules  are  a  near-term  possibility,  but  that  a  considerable  amount 
of  additional  work  would  be  required  to  develop  the  trade-off 
procedures . 

The  Crew  Chief  and  Profile  models  represent  major  efforts  in 
developing  maintainability  design  CAD  models.  Further  develop¬ 
ments  in  these  models  and  the  current  work  sponsored  by  the  Air 
Force  to  develop  expert  system/CAD  programs  for  improving  main¬ 
tainability  design  should  be  monitored  closely. 


Review  Literature  on  Troubleshooting  Performance 

With  the  possible  exception  of  physically  accessing  defective 
equipment,  troubleshooting  (fault  detection  and  isolation) 
typically  takes  up  the  majority  of  a  maintenance  technician's  time 
(see  Section  II) .  Therefore,  equipment  should  be  designed  so  that 
the  maintainer  with  the  skill  level  and  experience  expected  in  the 
field  can  easily  determine  the  presence,  nature,  and  location  of 
system  faults  and  determine  the  appropriate  course  of  action  (U.S. 
Air  Force  Systems  Command,  1988)  .  This  requires  a  thorough 
understanding  of  the  factors  that  affect  troubleshooting 
performance  (Morris  &  Rouse,  1984). 
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Although  the  reliability  of  BITE  has  increased  somewhat  in 
recent  years,  a  sizable  number  of  military  aviation  system  faults 
are  not  accurately  detected  and  isolated  by  BITE  and,  therefore, 
require  manual  troubleshooting  by  a  maintenance  technician 
(Frederickson,  Linquist,  &  Lehman,  1986) .  A  great  deal  of 
research  has  been  conducted  to  study  human  troubleshooting 
performance  (e.g..  Bond,  1987;  Rasmussen  &  Rouse,  1981)  and  to 
incorporate  the  findings  of  this  research  into  maintainability 
design  (e.g.,  Towne  et  al . ,  1988).  The  ultimate  goal  of  this 
research  is  to  identify  design  features  that  minimize  trouble¬ 
shooting  time  and  reduce  the  probability  of  false  removals  and 
replacements . 

Artificial  intelligence  (AI)  technology  is  playing  an 
increasingly  important  role  in  aviation  maintenance  trouble¬ 
shooting,  most  predominantly  in  the  form  of  expert  systems. 

Expert  systems  are  being  evaluated  as  maintenance  job  performance 
aids  by  the  Army,  the  Air  Force,  and  the  Navy.  Much  of  this  work 
was  summarized  during  the  Artificial  Intelligence  in  Maintenance 
Workshop  sponsored  by  the  Department  of  Defense  (Richardson  et 
al . ,  1985).  Reis  and  Thompson  (1988)  recently  described  three 
Army  Aviation  Systems  Command  (AVSCOM)  programs  that  are  exploring 
the  potential  benefit  of  expert  systems  for  troubleshooting 
specific  aircraft  systems  on  the  Army's  CH-47D  cargo  helicopter, 
AH-1S  attack  helicopter,  and  AH-64  advanced  attack  helicopter.  In 
addition,  work  is  in  progress  to  identify  the  requirements  for 
(a)  allocating  maintenance  functions  between  the  human  maintainer 
and  the  expert  system  and  (b)  designing  the  maintainer-expert 
system  interface  (e.g.,  Jonsson,  1988). 


Summary 

In  summary,  the  Crew  Chief  and  Profile  models  should  be 
evaluated  to  determine  their  utility  for  improving  the  maintain¬ 
ability  design  of  Army  aviation  systems.  The  MAPPS  maintainer 
model  and  the  HOS-IV,  Micro-SAINT,  SWAS  and  TAWL  operator  models 
should  be  evaluated  to  determine  the  desirability  and  feasibility 
of  modifying  them  (a)  to  simulate  the  performance  of  maintainer 
tasks  not  addressed  adequately  by  either  the  Crew  Chief  or  Profile 
models  and  (b)  to  predict  maintainer  workload. 

In  addition,  a  program  of  maintainability  research  should  be 
initiated  that  will  address  Army  aviation  systems,  mission 
requirements,  and  operational  environments.  This  program  should 
consist  of  the  following  activities: 

•  conducting  maintenance  task  and  workload  analyses  for 
selected  Army  aviation  systems  and  equipment  as  a  precursor 
to  the  development  of  a  model  of  Army  aviation  maintainer 
performance  and  workload; 

•  monitoring  research  and  development  efforts  to  develop 
expert  system/CAD  maintainability  design  models  that  could 
apply  to  Army  aviation  systems;  and 
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•  conducting  a  comprehensive  review  of  the  literature  on 
troubleshooting  performance,  especially  as  it  applies  to 
Army  aviation  systems,  and  monitoring  ongoing  research  and 
development  activities  in  this  area. 
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